Hardware fitting assembly safety test method and system based on artificial intelligence

Abstract

The invention relates to the technical field of data processing, and discloses a hardware fitting assembly safety test method and system based on artificial intelligence. The method comprises the steps of building a database by scanning and shooting hardware fittings, training a deep learning model to evaluate safety, deploying a test line to collect data of a real object and an operator, calculating an operation deviation, normalizing the data to calculate a safety score, and fusing the database, a deviation value and the safety score to generate a test result. According to the method, comprehensive evaluation of the characteristics of the hardware fitting, the behavior safety of the operator and the suitability of the hardware fitting and the operator is realized, so that potential safety hazards are accurately identified.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an artificial intelligence-based hardware accessory assembly safety testing method and system. Background Art

[0002] With the widespread application of hardware accessories in fields such as construction, furniture, and machinery, their assembly safety is becoming increasingly critical in product quality control. Traditional hardware assembly safety testing relies primarily on manual inspection and simple mechanical measurement methods, such as visual inspection, manual torque testing, and simple measuring tools. This method, typically performed by experienced quality inspectors, makes subjective judgments based on pre-set safety standards and, to a certain extent, meets basic safety requirements. With the advancement of industrial automation, some companies have begun employing single-sensor monitoring and simple pattern recognition technologies for safety testing. These include using force sensors to monitor assembly forces, visual sensors to detect surface defects, and pre-set test benches to measure the physical performance parameters of hardware accessories. These testing methods primarily focus on the physical properties and surface quality of the hardware accessories themselves, providing a preliminary assessment of their safety performance in actual assembly and usage environments.

[0003] However, existing safety testing methods for hardware assembly have significant shortcomings. First, traditional manual inspection methods are highly subjective and inefficient, making them difficult to meet the diverse and large-volume production demands of hardware accessories. Second, single-dimensional mechanical measurement or sensor testing cannot fully assess the comprehensive safety performance of accessories in complex usage environments. Third, existing testing methods lack an assessment of the safety of operator behavior, ignoring the important role of human-computer interaction in the assembly process. Fourth, scattered test data is difficult to effectively integrate and analyze, preventing the formation of systematic safety knowledge accumulation. Finally, the lack of intelligent safety risk warning and optimization suggestion mechanisms makes preventive safety management impossible. These shortcomings result in the continued existence of safety hazards in the actual use of hardware accessories, affecting product quality and user experience, and may even cause safety accidents. Therefore, there is an urgent need for an innovative method that can comprehensively utilize artificial intelligence technology to achieve multi-dimensional, comprehensive, and intelligent safety testing. Summary of the Invention

[0004] This application provides an artificial intelligence-based hardware accessories assembly safety testing method and system, which is used to achieve a comprehensive evaluation of the hardware accessories' own characteristics, the operator's behavioral safety, and the adaptability of the two, so as to accurately identify potential safety hazards and automatically generate targeted safety optimization strategies.

[0005] In a first aspect, the present application provides an artificial intelligence-based hardware accessory assembly safety testing method, which includes: scanning and photographing hardware accessory samples to obtain geometric data and image data, and establishing a hardware accessory safety feature database; training a deep learning safety assessment model based on the hardware accessory safety feature database, extracting and classifying features of the hardware accessories, and obtaining a safety status determination result; and deploying a safety test cell line on the production line based on the safety assessment model to collect images and physical parameter data of the hardware accessories; The safety test cell line is used to collect operator action data, the action sequence is analyzed through a feature extraction algorithm, and the deviation value from the safety operation standard is calculated; the data collected by the safety test cell line and the deviation value are normalized, and the safety adaptation score of the hardware accessories under the current operating conditions is calculated; the data of the safety feature database, the deviation value and the safety adaptation score are weighted and fused to generate the hardware accessories safety test results.

[0006] In a second aspect, the present application provides an artificial intelligence-based hardware accessories assembly safety testing system, the artificial intelligence-based hardware accessories assembly safety testing system comprising: Establish a module for scanning and photographing hardware accessories samples, obtaining geometric data and image data, and establishing a hardware accessories security feature database; An extraction module is used to train a deep learning safety assessment model based on the hardware accessories safety feature database, extract and classify the hardware accessories, and obtain a safety status determination result; An acquisition module is used to deploy a safety test cell line on the production line according to the safety assessment model to collect images and physical parameter data of hardware accessories; An analysis module, configured to collect operator action data using the safety test cell line, analyze the action sequence using a feature extraction algorithm, and calculate the deviation value from the safety operation standard; a processing module, configured to normalize the data collected by the safety test cell line and the deviation value, and calculate the safety adaptation score of the hardware accessories under the current operating conditions; The fusion module is used to perform weighted fusion calculation on the data of the security feature database, the deviation value and the safety adaptation score to generate a hardware accessory safety test result.

[0007] In a third aspect, an artificial intelligence-based hardware accessories assembly safety testing device is provided, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the artificial intelligence-based hardware accessories assembly safety testing device executes the above-mentioned artificial intelligence-based hardware accessories assembly safety testing method.

[0008] In a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, which, when executed on a computer, enables the computer to execute the above-mentioned artificial intelligence-based hardware accessories assembly safety testing method.

[0009] In the technical solution provided by the present application, by scanning and photographing hardware accessories samples to obtain geometric data and image data, a comprehensive hardware accessories safety feature database is established, which provides rich basic data support for safety assessment and solves the problem of inaccurate safety assessment caused by insufficient data in traditional methods; the deep learning safety assessment model trained based on the safety feature database can accurately extract and classify features of hardware accessories, significantly improves the accuracy of safety status judgment, and greatly reduces missed detections and misjudgments compared with traditional manual detection methods; the safety test cell line deployed on the production line realizes real-time collection of hardware accessories images and physical parameter data, establishes a dynamic monitoring mechanism in the production process, breaks through the limitations of traditional static detection methods, and realizes safety monitoring of the entire production process; by collecting operator action data and using feature extraction algorithms to analyze action sequences, calculate the deviation value from the safety operation standard, and innovatively incorporate human factors engineering factors into the safety assessment system, solving the shortcomings of traditional methods that only focus on products and ignore operating behaviors; The method of normalizing the data and deviation values of the set and calculating the safety adaptation score realizes the unified quantitative evaluation of multi-source heterogeneous data and improves the objectivity and comparability of the safety assessment; the data, deviation values and safety adaptation score of the security feature database are weighted and fused to generate the safety test results. In this solution, the ResNet-50 convolutional neural network, PointNet++ point cloud processing network, temporal convolutional network and self-attention mechanism are applied, combining the characteristics of different algorithms to effectively solve the multimodal data processing problem in the safety assessment of hardware accessories. At the same time, the application of the OpenPose skeleton detection algorithm and graph neural network enables the system to accurately capture the operator's action details and establish complex relationship models, providing a more comprehensive analysis dimension for safety assessment. The application of the decision tree algorithm in the generation of optimization strategies fully utilizes its strong interpretability and clear decision path to ensure the feasibility and pertinence of the safety optimization strategy, significantly improving the safety of the hardware assembly process, reducing the incidence of safety accidents, and improving production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 This is a schematic diagram of an embodiment of a hardware accessories assembly safety testing method based on artificial intelligence in an embodiment of the present application; Figure 2 This is a schematic diagram of an embodiment of a hardware accessories assembly safety testing system based on artificial intelligence in an embodiment of the present application; Figure 3 It is a schematic structural diagram of an artificial intelligence-based hardware accessories assembly safety testing device in an embodiment of the present invention. DETAILED DESCRIPTION

[0012] The embodiments of the present application provide a method and system for testing the safety of hardware accessories assembly based on artificial intelligence. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0013] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In the embodiment of the present application, an embodiment of the hardware accessories assembly safety testing method based on artificial intelligence includes: Step S101: Scan and photograph hardware accessory samples to obtain geometric data and image data, and establish a hardware accessory safety feature database; Step S102: Based on the hardware accessories safety feature database, a deep learning safety assessment model is trained to extract and classify the hardware accessories to obtain a safety status determination result; Step S103: deploying a safety test cell line on the production line according to the safety assessment model to collect images and physical parameter data of hardware accessories; Step S104: using the safety test cell line to collect operator action data, analyzing the action sequence through a feature extraction algorithm, and calculating the deviation value from the safety operation standard; Step S105: normalize the data and deviation values collected by the safety test cell line, and calculate the safety adaptation score of the hardware accessories under the current operating conditions; Step S106: Perform weighted fusion calculation on the data in the security feature database, the deviation value, and the security adaptation score to generate a hardware accessory safety test result.

[0014] It is understandable that the execution subject of this application can be a hardware accessories assembly safety testing system based on artificial intelligence, or a terminal or a server, which is not limited here. The embodiment of this application is described by taking the server as the execution subject as an example.

[0015] Specifically, when scanning and photographing hardware accessory samples, a 3D scanner performs a 360-degree scan of the hardware accessory, generating point cloud data with each point containing X, Y, and Z coordinate information. This point cloud data undergoes point cloud registration, mesh reconstruction, and feature extraction to form geometric data. Simultaneously, a high-definition industrial camera is used to photograph the hardware accessory from various angles, capturing images from the front, side, top, and bottom views. These images are pre-processed to extract edge features, texture features, and color histogram features, which are then stored as image data. For example, the geometric data for a door handle hardware item consists of approximately 10,000 points, describing key geometric features such as the handle's curvature and threaded hole locations. The image data includes visual features such as the coating condition and edge smoothness of the handle's surface. This data is categorized and organized by product type, material, and application to construct a structured hardware accessory security feature database.

[0016] A multimodal deep learning architecture is employed to train a deep learning safety assessment model based on a hardware accessories safety feature database. This model processes image data using a ResNet-50 convolutional neural network, comprised of five residual blocks, each consisting of three convolutional layers and skip connections, to extract hierarchical visual features. For geometric data, a PointNet++ network is used to process point cloud data, extracting shape features through hierarchical sampling and a multi-layer perceptron. Simultaneously, physical parameter data is processed using a temporal convolutional network to capture parameter variation patterns. These three features are fused using an attention mechanism, with weights calculated for each feature and a composite feature vector generated. For example, if a door handle is detected with a minor crack on its surface, while the geometric data indicates intact internal structure and the physical parameters indicate normal torsional strength, the model comprehensively assesses this as a low-risk hazard and issues a "Minor Defect - Pass" safety status determination. Based on the trained safety assessment model, multiple test units are deployed on the production line. Each unit incorporates a high-definition camera array, a set of physical parameter sensors, and a control module. Each test unit is responsible for safety testing a specific assembly step. As door lock components pass through the test area, high-definition cameras capture images of the components from multiple angles, while sensors measure the force, torque, and displacement generated during the assembly process. For example, during the door lock cylinder test, cameras observe the spring return state of the cylinder, while sensors record the friction and torque during key insertion and removal. This data is transmitted to a data processing server via Industrial Ethernet, forming a real-time test dataset.

[0017] The safety test cell line has expanded its functionality to collect operator motion data, adding dedicated operation monitoring cameras. These cameras capture the operator's movements while assembling hardware accessories. Using the OpenPose skeleton detection algorithm, the operator's key joint positions are identified, and a human posture skeleton model is constructed. This model contains 15-25 key points, describing the spatial positions of the operator's arms, torso, and other parts. This skeleton data is fed into a spatiotemporal graph convolutional network, which treats the human skeleton as a graph structure. Graph convolution extracts spatial features and temporal convolution captures motion sequence features, thereby identifying the type of operation. This data is compared with the safe operating postures defined in standard operating procedures, and a dynamic time warping algorithm is used to calculate sequence similarity, resulting in a deviation value. In a real-world case study, when an operator was assembling a door hinge, the detected wrist angle and force direction deviated by 30% from the standard safe posture. The system then determined a high-risk deviation. The collected data and deviation values are normalized to calculate a safety fit score. Min-Max normalization is applied to image data, mapping pixel values to the range [-1, 1]. Z-score normalization is applied to physical parameter data, ensuring a mean of 0 and a standard deviation of 1. Weighted normalization is performed on motion deviation values, mapping them to the range [0, 1]. A graph neural network model is then constructed, using hardware accessory data and motion deviation values as node features to establish correlations between accessory characteristics and operational behaviors. A message passing algorithm is used to exchange information between nodes and calculate the safety compatibility between the hardware accessory and the operational method. For the door hinge example, the system calculates a safety compatibility score of 0.65 for the current operation and this hinge model, placing it at a medium safety level. A weighted fusion calculation is performed to generate safety test results and automatically generate an optimization strategy. Historical data with high similarity to the test accessory is extracted from the security feature database, and the correlation coefficient between the historical safety assessment results and the current motion deviation value is calculated. Based on the correlation coefficient, the safety compatibility score, and pre-set weights, a weighted fusion formula is constructed to generate comprehensive safety test results.

[0018] In the embodiment of the present application, by scanning and photographing hardware accessories samples to obtain geometric data and image data, a comprehensive hardware accessories safety feature database is established, which provides rich basic data support for safety assessment and solves the problem of inaccurate safety assessment caused by insufficient data in traditional methods; the deep learning safety assessment model trained based on the safety feature database can accurately extract and classify features of hardware accessories, significantly improves the accuracy of safety status judgment, and greatly reduces missed detections and misjudgments compared with traditional manual detection methods; the safety test cell line deployed on the production line realizes real-time collection of hardware accessories images and physical parameter data, establishes a dynamic monitoring mechanism in the production process, breaks through the limitations of traditional static detection methods, and realizes safety monitoring of the entire production process; by collecting operator action data and using feature extraction algorithms to analyze action sequences, calculate the deviation value from the safety operation standard, and innovatively incorporate human factors engineering factors into the safety assessment system, which solves the shortcomings of traditional methods that only focus on products and ignore operating behaviors; The data and deviation values are normalized and the method of calculating the safety adaptation score is adopted, which realizes the unified quantitative evaluation of multi-source heterogeneous data and improves the objectivity and comparability of the safety assessment; the data, deviation values and safety adaptation scores of the security feature database are weighted and fused to generate the safety test results. In this solution, ResNet-50 convolutional neural network, PointNet++ point cloud processing network, temporal convolutional network and self-attention mechanism are applied, combining the characteristics of different algorithms to effectively solve the multimodal data processing problem in the safety assessment of hardware accessories; at the same time, the application of OpenPose skeleton detection algorithm and graph neural network enables the system to accurately capture the operator's action details and establish complex relationship models, providing a more comprehensive analysis dimension for safety assessment; the application of decision tree algorithm in optimization strategy generation fully utilizes its strong interpretability and clear decision path to ensure the feasibility and pertinence of safety optimization strategy, significantly improve the safety of hardware accessories assembly process, reduce the incidence of safety accidents, and improve production efficiency.

[0019] In a specific embodiment, the process of executing step S101 may specifically include the following steps: Use a 3D scanner to perform 360-degree scanning on hardware accessory samples to obtain 3D geometric data; Photograph the hardware accessories sample from four angles: front, side, top, and bottom, to obtain image data; Mark standard features of hardware accessories samples under normal conditions and record key dimensional parameters; Mark abnormal features of hardware accessories samples in damaged, deformed, or defective states, and record the characteristics of abnormal areas; Collect the assembly force, assembly angle, and tightening torque parameters of hardware accessories during the assembly process; The three-dimensional geometric data, image data, key dimensional parameters, abnormal area features and assembly parameters are classified and organized to build a hardware accessories safety feature database.

[0020] Specifically, a 3D scanner is used to perform a 360-degree scan of hardware accessories samples to obtain three-dimensional geometric data. A 3D scanner is a device that can capture three-dimensional information on the surface of an object. It uses laser or structured light technology to scan hardware accessories from multiple angles to generate point cloud data. These point cloud data contain a large number of spatial coordinate points, each of which consists of three coordinate values: X, Y, and Z, accurately recording the geometric shape and size of the hardware accessories. The scanning process needs to ensure that all surfaces of the hardware accessories are captured, especially key functional areas such as complex structural parts such as threads and grooves. Taking door lock hardware as an example, at least 8 different scanning angles will be set during the 3D scanning process. Approximately 10,000 points will be collected at each angle and merged to form a 3D point cloud model. The point cloud data is then aligned and fused using a point cloud registration algorithm to form 3D geometric data.

[0021] Hardware accessory samples were photographed from four angles: front, side, top, and bottom, to capture image data. High-resolution industrial cameras were used with standard lighting to ensure image clarity and consistent lighting. Images captured from each angle achieved a resolution of 4000 x 3000 pixels, providing sufficient detail for subsequent feature extraction. These images were processed for white balance, distortion correction, and dimension calibration to ensure image data accuracy. For example, the front image of a cabinet door hinge clearly depicts the hinge's overall appearance and connection structure, the side image shows its thickness and lateral features, and the top and bottom views reveal the hinge's top and bottom structures, respectively. Standard feature annotation was performed on the hardware accessory samples in their normal state, recording key dimensional parameters. This annotation work was performed by industry experts using specialized annotation tools. Key functional dimensions, tolerance ranges, and material properties were annotated for each hardware type. For example, for a window handle, key dimensional parameters such as handle length, rotation angle limit, and connecting shaft diameter were annotated, and the parameter values under standard operating conditions were recorded. The annotation data is stored in JSON format, and a standard feature file is generated for each hardware accessory, which contains information such as parameter name, value, unit, and allowable deviation range.

[0022] Abnormal feature annotation is performed on hardware accessories samples in damaged, deformed, and defective states, and the features of the abnormal areas are recorded. To obtain abnormal feature data, it is necessary to simulate different types of safety hazards, such as artificial cracks, wear, deformation, and material defects. Annotators use bounding boxes and segmentation masks to mark abnormal areas and classify abnormalities according to the degree of safety risk (minor, moderate, severe). The annotation information includes the type of abnormality, location coordinates, area size, and severity. Taking drawer slides as an example, abnormal features such as track deformation areas, ball bearing wear areas, and screw hole cracks are annotated. These abnormal feature data are compared with the normal state data to provide positive and negative samples for model training.

[0023] The hardware accessories' assembly force, assembly angle, and tightening torque parameters are collected during the assembly process. This process uses specialized force sensors, angle sensors, and torque sensors to test the hardware accessories under standard assembly processes. The sensors collect data at a 100Hz frequency, recording the physical parameter curves throughout the entire assembly process. For example, during the door handle installation phase, the force curves, installation angle curves, and tightening torque values were recorded for the insertion, rotation, and fixation stages. These physical parameters reflect the mechanical properties and safety performance of the hardware accessories in actual use. 3D geometric data, image data, key dimensional parameters, abnormal area features, and assembly parameters are categorized and organized to construct a hardware accessory safety feature database. The database design utilizes a relational database structure, establishing product information tables, geometry data tables, image data tables, standard feature tables, abnormal feature tables, and physical parameter tables, with product IDs used to establish relationships. The database also includes search indexes to enable rapid retrieval of relevant data based on accessory type, material, and application. For example, a high-end door lock's complete data includes a 3D geometric model consisting of 120,000 points, high-definition images from four angles, 15 key dimensional parameters (such as cylinder diameter and bolt extension distance), three types of typical abnormality records (cylinder wear, bolt deformation, and spring failure), and time series data on assembly physical parameters. This data is structured and stored in a database.

[0024] In a specific embodiment, the process of executing step S102 may specifically include the following steps: Image data is extracted from the hardware accessories security feature database and input into the ResNet-50 convolutional neural network. After processing through five residual blocks and a global average pooling layer, the visual features of the hardware accessories are obtained. Extract geometric data from the hardware accessories security feature database and input it into the PointNet++ point cloud processing network. Through hierarchical sampling and multi-layer perceptron processing, the shape features of the hardware accessories are obtained. Physical parameter data is extracted from the hardware accessories safety feature database and input into a temporal convolutional network with 8 convolutional layers to obtain the temporal features of the hardware accessories. The visual features, shape features and temporal features are integrated through the self-attention mechanism, and the weight coefficient of each feature is calculated to obtain a comprehensive feature vector; The comprehensive feature vector is input into a three-layer fully connected neural network, with 256 neurons in the first layer, 128 neurons in the second layer, and the third layer output layer corresponding to the number of safety status categories, to obtain the safety status classification result; The cross-entropy loss function is used to calculate the error between the model prediction value and the true label, and the parameters are updated through the Adam optimizer to obtain the trained deep learning security assessment model.

[0025] Specifically, image data is extracted from the hardware accessories security feature database and input into the ResNet-50 convolutional neural network for processing. ResNet-50 is a deep residual network that consists of 5 residual blocks, each of which contains multiple convolutional layers and jump connection structures. After the image data of hardware accessories is input into the network, it undergoes preliminary feature extraction through a 7×7 convolutional layer and a maximum pooling layer, and then passes through 5 residual blocks in sequence. The jump connection in each residual block allows information to be passed directly from the shallow layer to the deep layer, effectively preserving the original features. Finally, the two-dimensional feature map is compressed into a one-dimensional vector through a global average pooling layer to obtain the visual features of the hardware accessories. Specifically, for images of door lock accessories, the visual features extracted by ResNet-50 contain key visual information such as the lock core structure, lock tongue shape, and surface texture. These features are encoded into a 2048-dimensional feature vector.

[0026] At the same time, geometric data is extracted from the security feature database and input into the PointNet++ point cloud processing network. PointNet++ is a deep learning network specifically designed to process three-dimensional point cloud data. It handles irregular point cloud data through hierarchical sampling and local feature aggregation. During processing, PointNet++ performs multi-scale sampling to decompose the original point cloud data (such as the complete point cloud model of a door hinge) into multiple local regions. A spherical neighborhood search is then applied to each local region to extract local features. These local features are transformed and aggregated using a multi-layer perceptron. The point cloud resolution is then restored through hierarchical upsampling to generate feature vectors that can describe the geometric shape of the hardware accessories. For example, for slide rail accessories, PointNet++ can accurately capture key geometric features such as the track curvature, slider structure, and mounting hole location. This information is crucial for determining the structural safety of the accessories.

[0027] Physical parameter data is extracted from the database, including time-series information such as force changes, angle changes, and torque changes during the installation process. This time-series data is processed by a temporal convolutional network (TCN) consisting of eight convolutional layers. The TCN employs a one-dimensional convolutional architecture, which effectively captures patterns in time-series data. The first layer uses a large convolution kernel (e.g., 7×1) to capture long-term dependencies, while subsequent layers gradually reduce the kernel size to extract finer temporal features. Each convolution layer is followed by batch normalization and a Reluctant Unit (ReLU) activation function to ensure nonlinear representation. This architecture enables the TCN to extract key temporal features from the time-series data of physical parameters during door handle installation, such as peak force, torque stability, and angle variation patterns. Subsequently, the visual, shape, and temporal features are fused using a self-attention mechanism. The self-attention mechanism dynamically adjusts the importance of different features and is particularly important in hardware accessory safety assessment, as the safety of different accessory types may be primarily determined by different features. In its implementation, the three feature vectors are first mapped to a latent space of the same dimensionality through a linear transformation, and then a correlation score matrix is calculated between the features. Based on the relevance score, a weighting coefficient is calculated for each feature, and weighted fusion is performed. For example, for cabinet door handles, where the surface coating is a critical safety point, visual features are given a higher weight; while for load-bearing structural components such as brackets, shape features and physical parameter features are given higher weights. This dynamic weighted fusion ensures a specific safety assessment for each hardware accessory and generates a comprehensive feature vector.

[0028] The fused comprehensive feature vector is input into a three-layer fully connected neural network for safety status classification. The first layer of this network contains 256 neurons and receives the comprehensive feature vector as input. The second layer contains 128 neurons, which further extracts high-level abstract features. The third layer is the output layer, with the number of neurons matching the number of safety status categories (typically including "safe," "minor hazard," "moderate hazard," and "serious hazard"). ReLU activation functions are used between each layer to introduce nonlinearity, and dropout (with a dropout rate of 0.5) is applied to prevent overfitting. The third layer uses a softmax activation function to convert the output into a probability distribution, representing the probability of a sample belonging to each safety status category. The cross-entropy loss function is used to calculate the error between the model's prediction and the true label. The cross-entropy loss function is a standard loss function for multi-classification problems, measuring the difference between the predicted probability distribution and the true label distribution. During training, the model parameters are updated using the Adam optimizer. The Adam optimizer combines the advantages of momentum gradient descent and an adaptive learning rate, making it suitable for handling high-dimensional parameter spaces and sparse gradients. The training process uses a batch approach with 64 samples per batch. The learning rate is initially set to 0.001 and is decayed by 0.1 every 50 epochs. The entire training process lasts for 200 epochs or until performance on the validation set stops improving. Early stopping is used to avoid overfitting.

[0029] In a specific embodiment, the process of executing step S103 may specifically include the following steps: Multiple test cells are divided on the hardware accessories production line. Each test cell is equipped with 4 high-definition cameras to form a safe test cell line structure and a test area. Install force sensors, torque sensors, and displacement sensors in the test area to form a physical parameter acquisition system; Set image acquisition parameters based on the security assessment model, including acquisition angle, resolution, and frame rate, to form an image acquisition configuration; According to the image acquisition configuration, the hardware accessories passing through the test area are photographed from multiple angles to obtain real-time image data; Use the physical parameter acquisition system to collect the force, torque and displacement data generated during the assembly of hardware accessories to obtain physical parameter data; The real-time image data and physical parameter data are transmitted to the data processing server via industrial Ethernet to form a safety test data set.

[0030] Specifically, multiple test units are divided on the hardware accessories production line, and each unit is responsible for the safety testing of a specific type of hardware accessories. The safety test cell line is a distributed testing architecture that divides the traditional assembly line into independent but collaborative test units. Each test unit is equipped with four high-definition cameras to form a test area. These four cameras are installed above, in front, on the left and on the right side of the test area to ensure all-round image acquisition of hardware accessories without blind spots. The camera uses an industrial-grade high-definition camera with a resolution of 4K. It has autofocus and lighting compensation functions and can adapt to different test environments. Each test unit is responsible for a specific assembly link on the production line. For example, for a door lock production line, a lock body test unit, a lock cylinder test unit and a lock tongue test unit can be set up.

[0031] Force sensors, torque sensors, and displacement sensors are installed in each test area to form a physical parameter acquisition system. Force sensors are typically fixed to the base or side of the assembly station and are used to measure parameters such as pressure and tension during the assembly process. Torque sensors are installed on rotating assembly tools to measure torque changes during the tightening process. Displacement sensors monitor component movement distance and position changes. These sensors are high-precision industrial-grade equipment, with force sensors achieving an accuracy of ±0.1N, torque sensors achieving an accuracy of ±0.01N·m, and displacement sensors achieving an accuracy of ±0.01mm. Sensor data is converted to digital signals via an analog-to-digital converter, with a sampling frequency set to 100Hz to ensure that even brief changes in force and displacement are captured.

[0032] Based on the previously trained safety assessment model, the image acquisition parameters of each test unit are set. Image acquisition parameters are key configurations that guide the camera's operation, including acquisition angle, resolution, and frame rate. The acquisition angle is determined by the geometric characteristics of the hardware accessories. Different camera positions and shooting angles are preset for different types of accessories to ensure that the key features of the accessories are captured. For example, for door hinges, the upper camera focuses on the hinge shaft, while the side camera focuses on the structure of the connecting plate. The resolution setting directly affects the image's ability to capture details and is usually configured to 3840×2160 pixels to ensure that millimeter-level surface defects can be identified. The frame rate determines the temporal resolution and is usually set to 5 frames / second for static testing, while it is increased to 30 frames / second for dynamic assembly testing. These parameters are uniformly configured through the control software to form an image acquisition configuration plan.

[0033] Based on the configured image acquisition configuration, hardware accessories passing through the test area are captured from multiple angles, acquiring real-time image data. When a hardware accessory enters the test area, a position sensor triggers the camera to simultaneously capture images of the accessory from four angles. The raw image data undergoes preprocessing, including illumination correction, distortion correction, and background subtraction, followed by feature extraction and enhancement. For example, in the case of a door handle, the image processing algorithm automatically identifies the handle's contours, surface, and joints, capturing close-up images and enhancing these areas. The processed image data is timestamped and positionally identified for easy synchronization with physical parameter data. A physical parameter acquisition system is used to collect force, torque, and displacement data generated during the assembly of hardware accessories. Physical parameter acquisition is a crucial tool for evaluating the mechanical properties of hardware accessories, particularly load-bearing and connection-related hardware. As operators assemble hardware accessories, force sensors record the applied pressure curve, torque sensors monitor rotational tightening force, and displacement sensors track component movement. This physical data reflects the mechanical response characteristics of the accessory in real-world use. The collected raw sensor signals are filtered to remove noise and then normalized to convert different physical quantities to the same numerical range for subsequent analysis. For door lock installation, a complete physical parameter dataset is recorded for key operations such as inserting the lock body, rotating the lock cylinder, and pushing and pulling the lock tongue.

[0034] Real-time image data and physical parameter data are transmitted to a data processing server via Industrial Ethernet to form a secure test data set. Industrial Ethernet is a high-reliability network technology used in industrial environments, offering low latency and high anti-interference capabilities. Data transmission utilizes real-time Ethernet protocols such as EtherCAT or Profinet, with transmission rates reaching 100Mbps, ensuring real-time transmission of large amounts of image and sensor data. Once the data arrives at the server, it undergoes time synchronization, aligning data from different devices according to timestamps. Data fusion is then performed, linking the image data and physical parameter data to form a structured test record. Each record contains information such as the hardware accessory ID, test time, image data from four angles, force curve, torque data, and displacement trajectory.

[0035] In a specific embodiment, the process of executing step S104 may specifically include the following steps: Add operation monitoring cameras to the safety test cell line to capture video of operators assembling hardware accessories and obtain operation video sequences; Apply the OpenPose skeleton detection algorithm to the operation video sequence, extract the coordinates of the operator's key joints, and construct a human posture skeleton model; The coordinate sequence of the human posture skeleton model is input into the spatiotemporal graph convolutional network, and processed through the graph convolution layer and the temporal convolution layer to identify the operator's action type; Extract standard operation skeleton sequence parameters corresponding to hardware accessories from the safe operation database as a reference standard for safe operation; The dynamic time warping algorithm is used to calculate the operator's action type and the safe operation reference standard to obtain the time-aligned action sequence; The Euclidean distance between the time-aligned action sequence and the safe operation reference standard in three dimensions: joint angle, action amplitude, and operation sequence is calculated to obtain the operation deviation value.

[0036] Specifically, operation monitoring cameras are added to the deployed safety test cell lines, focusing primarily on the operator rather than the components themselves. Typically, two to three cameras are installed in each test cell, covering the operator's work area from different angles to ensure accurate motion capture. These cameras use high-definition wide-angle lenses, a frame rate of 30 fps, and a resolution of 1920 × 1080 pixels to ensure accurate motion capture. As the operator assembles hardware components at the workstation, the cameras continuously capture video data, generating an operation video sequence with a duration consistent with the assembly process. For example, in the case of door lock installation, the entire process, from picking up the lock, positioning, drilling, to final fastening, is recorded. After acquiring the operation video sequence, the OpenPose skeleton detection algorithm is applied to it. OpenPose is a computer vision technology for human pose estimation that automatically identifies the locations of key human joints from two-dimensional images. In this processing, each frame undergoes preprocessing, including resizing and pixel normalization. Human key points are then detected using a multi-stage convolutional neural network, which comprises a feature extraction stage and a key point prediction stage. Feature extraction uses the VGG-19 network structure to generate feature maps; key point prediction is generated through a series of convolutional layers and confidence maps to identify the coordinates of 15-25 key points including the head, neck, shoulders, elbows, wrists, hips, knees and ankles. For hardware assembly scenarios, the recognition accuracy of upper limb joints (shoulders, elbows, wrists, fingers) is particularly enhanced because these parts are the most critical in assembly operations. Each joint point consists of an (x, y) coordinate value and a confidence score. Detection points with a confidence score below 0.6 will be filtered out to ensure data quality. By connecting these key points, a skeleton model representing the human body structure is constructed. This model records the operator's posture changes during the assembly process in the form of a time series.

[0037] The constructed human posture skeleton model data requires further analysis, so the skeleton coordinate sequence is fed into a spatiotemporal graph convolutional network (ST-GCN). ST-GCN is a deep learning architecture specifically designed for processing skeleton sequence data. It views the human skeleton as a graph structure, with nodes representing joints and edges representing skeletal connections. During data processing, the skeleton sequence is converted into a standard graph representation, where each node contains coordinate and velocity information. A graph convolution layer then extracts spatial features to capture the spatial relationships between joints. This graph convolution operation weights the features of each node with those of its neighbors, with the weights determined by the connectivity between nodes. Subsequently, a temporal convolution layer processes feature variations along the temporal dimension to capture the temporal patterns of the action. Global pooling and fully connected layers map the extracted features to predefined action categories, such as "correct tightening," "incorrect force," and "dangerous posture." Through multi-layer processing, the ST-GCN network gradually extracts high-level features of the action, ultimately outputting a classification result for the operator's action type. To assess the safety of the operation, standard skeleton sequence parameters corresponding to the hardware accessory in question are extracted from a safe operation database. The safe operation database is a library of standard operating behaviors defined and verified by industry experts, categorized and stored by hardware accessory type and assembly process. Each hardware accessory has a corresponding standard operation sequence, including parameters such as correct body posture, hand movements, force control, and operation rhythm. For example, for door hinge installation, the standard operation requires maintaining a straight back, a neutral wrist position, and a two-step fixation method. These standards are quantified into skeleton model parameters, including numerical indicators such as joint angle range, movement trajectory, and velocity curve. The extraction process uses accessory type and operation type as indexes to obtain the corresponding standard operation skeleton sequence from the database, which serves as a reference benchmark for evaluating operation safety.

[0038] Because actual operations often differ from standard operations in duration and tempo, a dynamic time warping (DTW) algorithm is used for sequence alignment. DTW is an algorithm that measures the similarity between two time series and is capable of handling sequences of varying speeds and durations. During the processing process, a distance matrix, typically using Euclidean distance, is calculated between each time point in the actual operation sequence and the standard sequence. A dynamic programming algorithm is then used to find the optimal matching path, representing the mapping relationship between corresponding points in the two sequences and minimizing the total distance. DTW allows for nonlinear scaling of the time axis and can handle uneven operation speeds. After DTW processing, a time-aligned operation sequence is obtained, ensuring a one-to-one correspondence between the actual and standard operations in time, facilitating subsequent accurate comparisons. The differences between the time-aligned action sequence and the reference safety operation standard are calculated in multiple dimensions to determine the operation deviation value. The specific calculation process is divided into three dimensions: the joint angle dimension calculates the difference between the actual operation and the standard operation in each key joint angle. The joint angle is calculated by calculating the angle between the vectors determined by three points; the movement amplitude dimension calculates the difference in amplitude between the actual movement and the standard movement. The amplitude is expressed as the displacement distance of the joint points; the operation sequence dimension evaluates whether the sequence of operation steps conforms to the standard process. The difference values of the three dimensions are calculated using the Euclidean distance formula, which is the square root of the sum of the squares of the coordinate differences of each corresponding point. Finally, the distance values of the three dimensions are weighted and combined to obtain a comprehensive operation deviation value, which reflects the degree of compliance of the operator's actions with safety standards.

[0039] In a specific embodiment, the process of executing step S105 may specifically include the following steps: Perform Min-Max normalization on the hardware accessories image data collected by the safety test cell line. If the image pixel value exceeds the preset threshold, the excess part is truncated to obtain normalized image data; The physical parameter data collected by the safety test cell line were normalized by Z-score. If the normalized value deviated from the mean by more than 3 standard deviations, it was identified as an outlier and replaced with a boundary value to obtain the normalized physical parameter data. The deviation value is weighted according to the importance of the joint. If the joint deviation value is greater than the safety threshold, the weight coefficient of the corresponding joint is increased by 50% to obtain the weighted deviation value; The normalized image data, standardized physical parameter data, and weighted deviation values are input into the graph neural network to construct a ternary relationship graph. If the correlation between nodes is lower than 0.3, the connection is disconnected to obtain an optimized relationship graph. Execute the message passing algorithm on the optimized relationship graph to calculate the information flow between nodes. If the iteration fails to converge after more than 10 times, it is forced to terminate and the last result is used to obtain the node feature vector. Based on the node feature vector, the Sigmoid function is applied to map the calculation result to the interval [0,1]. If the result is less than 0.6, it is judged as low safety adaptation. If the result is greater than 0.8, it is judged as high safety adaptation. The safety adaptation score of the hardware accessories under the current operating conditions is obtained.

[0040] Specifically, the hardware accessory image data collected by the safety test cell line undergoes Min-Max normalization, a commonly used data normalization method that maps image pixel values of varying scales to a uniform interval for ease of subsequent analysis and processing. This involves linearly transforming the original image's pixel values according to their maximum and minimum values, converting them to the interval [-1, 1]. For RGB images, each color channel is processed separately. During processing, if certain pixel values are abnormally high or low, exceeding a preset threshold (typically set at three standard deviations of the original data distribution), truncation is performed to limit the excess value to within the threshold, thus preventing outliers from interfering with subsequent analysis. For example, when inspecting door handle surface coatings, high-reflectivity areas can produce overexposed pixel values. Truncation limits these values to a reasonable range, resulting in more balanced and clear texture features and surface details in the image.

[0041] Physical parameter data collected from the safety test cell line undergoes Z-score normalization. Physical parameter data includes measurements in various units and magnitudes, such as force, torque, and displacement. Normalization is essential for consistent comparison and analysis. Z-score normalization transforms raw data into a standard normal distribution with a mean of 0 and a standard deviation of 1. This is calculated by subtracting the mean from the raw value and then dividing by the standard deviation. In practice, the mean and standard deviation of each physical parameter are first calculated, and then each data point is normalized. After normalization, outliers are identified by determining whether the value deviates from the mean by more than 3 standard deviations. This threshold is based on the statistical "3σ rule," which states that 99.7% of data in a normal distribution fall within the range of ±3 standard deviations from the mean. Data points identified as outliers are not simply deleted but replaced with borderline values (mean ±3 standard deviations). This approach eliminates the influence of extreme values while preserving the overall data structure. For example, in the torque data measured during door lock installation, if a particular moment in time shows an abnormally high torque value, this could be due to operator error or sensor failure. Z-score normalization and outlier processing minimize the impact of this outlier, resulting in a smoother and more realistic torque curve. The acquired operational deviation values are weighted according to joint importance. Different joints play varying roles in hardware assembly safety assessments. For example, the wrist and elbow are often the most vulnerable parts of assembly operations, and their deviations should be given higher weights. In implementation, a base weight coefficient is predefined for each joint based on ergonomics and safety standards. This weight coefficient is then dynamically adjusted based on actual detected deviations. If a joint's deviation exceeds a preset safety threshold (typically based on medical research and ergonomics), the joint is considered high-risk and its weight coefficient is increased by 50%, giving greater emphasis in the final score. This dynamic weighting mechanism ensures that the safety assessment focuses on operational actions that actually pose a high risk, rather than simply assigning an equal weight to all actions. Taking drawer slide installation as an example, if the system detects that the operator's wrist angle continues to exceed the safe range during a long installation process, the system will automatically increase the weight of wrist deviation in the total score, thereby generating a more sensitive and accurate safety warning.

[0042] The three types of processed data mentioned above—normalized image data, standardized physical parameter data, and weighted deviation values—are input into a graph neural network to construct a ternary relationship graph. A graph neural network is a deep learning model that specializes in processing graph-structured data and can effectively capture the complex relationships between different data. In this method, the ternary relationship graph treats the three types of data as different types of nodes, and the relationships between them are represented by edge connections. The specific construction process first defines the node types and features: image nodes contain visual feature vectors, physical parameter nodes contain time series feature vectors, and motion deviation nodes contain joint deviation values. Edge connections are then established between nodes, and the connection strength is calculated based on the correlation between the data. To prevent weakly correlated or irrelevant connections from affecting the analysis accuracy, a correlation threshold of 0.3 is set. For node pairs with correlations below this threshold, their connections are disconnected, resulting in an optimized relationship graph. This pruning process effectively reduces the complexity of the graph and improves the efficiency and accuracy of subsequent calculations. For example, when analyzing the assembly process of cabinet door hinges, subtle scratches on the hinge surface (image features) have almost no correlation with the assembly force (physical parameters), and the edge between the two can be safely removed. However, the position deviation of the hinge connection hole (image feature) is highly correlated with the operator's wrist angle (motion deviation), and its connection should be retained.

[0043] A message passing algorithm is executed on the constructed optimized relationship graph to calculate the information flow between nodes. Message passing is used to interact and update node features. The basic idea is that each node generates messages based on its own features and those of its neighboring nodes, and then aggregates these messages to update its own representation. The specific implementation process includes the following steps: initializing the feature vectors of all nodes; computing messages for each edge, where the message content is the result of processing the source node's features through a transformation function; aggregating all messages directed to each node, typically by summing or averaging them; updating the node's features based on the aggregated messages; and repeating these steps for multiple rounds. In practical applications, the maximum number of iterations is set to 10. If the algorithm fails to converge before reaching the maximum number of iterations (i.e., the change in node features is still greater than a preset threshold), the iteration process is terminated and the result of the last iteration is used as the final node feature vector. This approach ensures both computational efficiency and algorithm stability. For example, in analyzing hinge assembly, multiple rounds of message passing enable the operational action features to fully integrate component defect information and physical parameter anomaly patterns, forming a comprehensive representation of safety risks. Based on the node feature vector obtained after message transmission, the Sigmoid function is applied to map the calculated result to the interval [0,1] to obtain the safety adaptation score of the hardware accessories under the current operating conditions. The Sigmoid function can map any real number to the interval (0,1). In this method, the node feature vectors are weighted and summed to obtain the original safety score; then, the Sigmoid function is used to convert it into a safety adaptation score between 0 and 1, where the closer it is to 1, the higher the safety. According to industry standards and safety specifications, the judgment threshold of the safety adaptation score is pre-set: less than 0.6 is judged as low safety adaptation, indicating a high safety risk; greater than 0.8 is judged as high safety adaptation, indicating a low safety risk; between 0.6 and 0.8 is medium safety adaptation, which requires certain attention. This multi-level classification method avoids the roughness of simple dichotomy and provides more detailed risk assessment results.

[0044] In a specific embodiment, the process of executing step S106 may specifically include the following steps: Retrieve historical security data of the currently tested hardware accessories from the security feature database, extract the historical data subset with the highest similarity through cosine similarity calculation, and obtain the reference security feature; The reference safety feature, deviation value, and safety adaptation score were assigned initial weight coefficients of 0.3, 0.3, and 0.4, respectively, and a weighted calculation formula was constructed to obtain a three-factor fusion model. The calculation results of the three-factor fusion model are compared with the preset safety level threshold, and the test results are divided into four levels: safety, reminder, warning, and danger to obtain the hardware accessories safety test results.

[0045] Specifically, historical safety data for the hardware accessory under test is retrieved from the safety feature database. This process utilizes feature matching-based retrieval technology. The safety feature database is a structured database established in a previous step and contains a large number of safety feature records for hardware accessories. The retrieval process first extracts key features of the hardware accessory under test, including geometric, material, and functional characteristics. These features are then matched against historical records in the database. This comparison utilizes cosine similarity, a technique that measures the similarity between two vectors. This method calculates the cosine of the angle between the two feature vectors, ranging from -1 to 1, with values closer to 1 indicating greater similarity. Specifically, the cosine similarity is calculated between the feature vector of the current accessory and the feature vector of each record in the database. The records are then sorted from highest to lowest similarity, and a subset of records with a similarity exceeding a preset threshold (typically 0.8) is extracted as reference safety features. This similarity-based screening ensures that the selected historical data is highly relevant to the hardware accessory under test. A weighted fusion is then performed on the acquired reference safety features, the operator motion deviation value calculated in the previous step, and the safety adaptation score. These three types of data represent safety assessment results from different perspectives: the reference safety characteristics reflect the historical safety status of the accessory itself; the action deviation value reflects the standardization of the operator's behavior; and the safety adaptation score indicates the degree of compatibility between the accessory and the current operating environment. To comprehensively consider these three factors, they are assigned initial weight coefficients of 0.3, 0.3, and 0.4, respectively, and a weighted calculation formula is constructed. The weighting is based on the experience of industry experts and the analysis results of a large amount of test data. The safety adaptation score is slightly higher than the other two because it already incorporates interactive information between accessory characteristics and operating actions. The weighted calculation is achieved through linear combination. That is, the values of the three factors are multiplied by the corresponding weights and the sum is calculated to obtain a comprehensive safety score. This three-factor fusion model overcomes the limitations of single-dimensional evaluation and comprehensively considers all aspects that affect the safety of hardware accessory installation.

[0046] The comprehensive safety score calculated by the three-factor fusion model is compared with the preset safety level thresholds, and the test results are classified into different levels. Safety level thresholds are pre-defined cutoff points based on industry standards and safety regulations. Generally, 0.9-1.0 is defined as the "safe" level, indicating no obvious safety risks; 0.7-0.9 is defined as the "alert" level, indicating a minor safety risk requiring attention; 0.5-0.7 is defined as the "warning" level, indicating a moderate risk requiring prompt action; and 0-0.5 is defined as the "dangerous" level, indicating a serious safety risk requiring immediate intervention.

[0047] The above describes the hardware accessories assembly safety testing method based on artificial intelligence in the embodiment of the present application. The following describes the hardware accessories assembly safety testing system based on artificial intelligence in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of the hardware accessories assembly safety testing system based on artificial intelligence includes: Establishing module 201, for scanning and photographing hardware accessory samples, obtaining geometric data and image data, and establishing a hardware accessory safety feature database; An extraction module 202 is used to train a deep learning security assessment model based on the hardware accessories security feature database, extract and classify the hardware accessories, and obtain a safety status determination result; An acquisition module 203 is configured to deploy a safety test cell line on the production line according to the safety assessment model to collect images and physical parameter data of hardware accessories; An analysis module 204 is configured to collect operator action data using the safety test cell line, analyze the action sequence using a feature extraction algorithm, and calculate the deviation value from the safety operation standard; The processing module 205 is used to normalize the data collected by the safety test cell line and the deviation value, and calculate the safety adaptation score of the hardware accessories under the current operating conditions; The fusion module 206 is used to perform weighted fusion calculation on the data of the security feature database, the deviation value and the security adaptation score to generate a hardware accessory safety test result.

[0048] Through the collaborative cooperation of the above-mentioned components, by scanning and photographing hardware accessories samples, obtaining geometric data and image data, a comprehensive hardware accessories safety feature database was established, which provided rich basic data support for safety assessment and solved the problem of inaccurate safety assessment caused by insufficient data in traditional methods; the deep learning safety assessment model trained based on this safety feature database can accurately extract and classify features of hardware accessories, significantly improving the accuracy of safety status judgment, and greatly reducing missed detections and misjudgments compared with traditional manual inspection methods; the safety test cell line deployed on the production line realizes the real-time collection of hardware accessories images and physical parameter data, establishes a dynamic monitoring mechanism in the production process, breaks through the limitations of traditional static inspection methods, and realizes safety monitoring of the entire production process; by collecting operator action data and using feature extraction algorithms to analyze action sequences, calculate the deviation value from the safety operation standard, and innovatively incorporate human factors engineering factors into the safety assessment system, solving the shortcomings of traditional methods that only focus on products and ignore operating behaviors; The method of normalizing the data and deviation values collected online and calculating the safety adaptation score realizes the unified quantitative evaluation of multi-source heterogeneous data and improves the objectivity and comparability of the safety assessment; the data, deviation values and safety adaptation score of the security feature database are weighted and fused to generate the safety test results. This solution applies ResNet-50 convolutional neural network, PointNet++ point cloud processing network, temporal convolutional network and self-attention mechanism, combining the characteristics of different algorithms to effectively solve the multimodal data processing problem in the safety assessment of hardware accessories; at the same time, the application of OpenPose skeleton detection algorithm and graph neural network enables the system to accurately capture the operator's action details and establish complex relationship models, providing a more comprehensive analysis dimension for safety assessment; the application of decision tree algorithm in optimization strategy generation fully utilizes its strong interpretability and clear decision path to ensure the feasibility and pertinence of safety optimization strategy, significantly improve the safety of the hardware assembly process, reduce the incidence of safety accidents, and improve production efficiency.

[0049] above Figure 2 The artificial intelligence-based hardware accessories assembly safety testing system in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The artificial intelligence-based hardware accessories assembly safety testing equipment in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0050] Figure 3This is a schematic diagram of the structure of an AI-based hardware assembly safety testing device provided by an embodiment of the present invention. This AI-based hardware assembly safety testing device 300 can vary significantly depending on configuration or performance. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors), memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) storing application programs 333 or data 332. The memory 320 and storage media 330 may be either transient or persistent storage. The program stored in the storage medium 330 may include one or more modules (not shown), each of which may include a series of instructions and operations within the AI-based hardware assembly safety testing device 300. Furthermore, the processor 310 may be configured to communicate with the storage medium 330, allowing the AI-based hardware assembly safety testing device 300 to execute the series of instructions and operations stored in the storage medium 330 to implement the steps of the aforementioned AI-based hardware assembly safety testing method.

[0051] The hardware accessories assembly safety testing device 300 based on artificial intelligence may further include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be understood by those skilled in the art that Figure 3 The structure of the artificial intelligence-based hardware accessories assembly safety testing equipment shown does not constitute a limitation of the artificial intelligence-based hardware accessories assembly safety testing equipment provided by the present invention, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0052] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, and when the instructions are run on a computer, the computer executes the steps of the artificial intelligence-based hardware accessories assembly safety testing method.

[0053] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0054] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling an artificial intelligence-based hardware accessory assembly safety testing device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0055] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A hardware accessories assembly safety testing method based on artificial intelligence, characterized in that: The method comprises: Scan and photograph hardware accessories samples to obtain geometric data and image data, and establish a hardware accessories safety feature database; Based on the hardware accessories security feature database, a deep learning security assessment model is trained to extract and classify the features of the hardware accessories to obtain a safety status determination result; Based on the safety assessment model, a safety test cell line is deployed on the production line to collect images and physical parameter data of hardware accessories; The safety test cell line is used to collect operator action data, and the action sequence is analyzed by a feature extraction algorithm to calculate the deviation value from the safety operation standard; Normalizing the data collected by the safety test cell line and the deviation value, and calculating the safety adaptation score of the hardware accessories under the current operating conditions; The data of the safety feature database, the deviation value and the safety adaptation score are weightedly fused and calculated to generate a hardware accessory safety test result.

2. The hardware accessories assembly safety testing method based on artificial intelligence according to claim 1 is characterized in that: The scanning and photographing of hardware accessory samples to obtain geometric data and image data and establish a hardware accessory safety feature database includes: Use a 3D scanner to perform 360-degree scanning on hardware accessory samples to obtain 3D geometric data; The hardware accessory sample is photographed from four angles: front, side, top, and top, to obtain image data; Marking the hardware accessories samples with standard features in a normal state and recording key dimensional parameters; Mark abnormal features of the hardware accessories samples in the damaged, deformed, or defective state, and record the features of the abnormal areas; Collecting the assembly force, assembly angle, and tightening torque parameters of the hardware accessories during the assembly process; The three-dimensional geometric data, the image data, the key size parameters, the abnormal area features and assembly parameters are classified and sorted to construct a hardware accessories safety feature database.

3. The hardware accessories assembly safety testing method based on artificial intelligence according to claim 1 is characterized in that: Based on the hardware accessories safety feature database, Train a deep learning safety assessment model to extract and classify hardware accessories to obtain safety status determination results, including: Extracting image data from the hardware accessories security feature database, inputting it into a ResNet-50 convolutional neural network, and processing it through five residual blocks and a global average pooling layer to obtain visual features of the hardware accessories; Extracting geometric data from the hardware accessories security feature database, inputting it into the PointNet++ point cloud processing network, and obtaining the shape features of the hardware accessories through hierarchical sampling and multi-layer perceptron processing; Extracting physical parameter data from the hardware accessories safety feature database and inputting it into a temporal convolutional network containing 8 convolutional layers to obtain the temporal features of the hardware accessories; The visual features, the shape features, and the temporal features are integrated through a self-attention mechanism, and a weight coefficient of each feature is calculated to obtain a comprehensive feature vector; The comprehensive feature vector is input into a three-layer fully connected neural network, with 256 neurons in the first layer, 128 neurons in the second layer, and the third layer output layer corresponding to the number of safety status categories, to obtain the safety status classification result; The cross-entropy loss function is used to calculate the error between the model prediction value and the true label, and the parameters are updated through the Adam optimizer to obtain the trained deep learning security assessment model.

4. The hardware accessories assembly safety testing method based on artificial intelligence according to claim 1 is characterized in that: According to the safety assessment model, a safety test cell line is deployed on the production line to collect images and physical parameter data of hardware accessories, including: Divide the hardware accessories production line into multiple test units, each equipped with four high-definition cameras to form a safe test cell line structure and a test area; Installing a force sensor, a torque sensor, and a displacement sensor in the test area to form a physical parameter acquisition system; Setting image acquisition parameters based on the safety assessment model, including acquisition angle, resolution, and frame rate, to form an image acquisition configuration; According to the image acquisition configuration, hardware accessories passing through the test area are photographed from multiple angles to obtain real-time image data; Using the physical parameter acquisition system, the force, torque and displacement data generated during the assembly of hardware accessories are collected to obtain physical parameter data; The real-time image data and the physical parameter data are transmitted to a data processing server via industrial Ethernet to form a safety test data set.

5. The hardware accessories assembly safety testing method based on artificial intelligence according to claim 1 is characterized in that: The method of collecting operator action data using the safety test cell line, analyzing the action sequence using a feature extraction algorithm, and calculating the deviation value from the safety operation standard includes: An operation monitoring camera is added to the safety test cell line to capture video of the operator assembling hardware accessories to obtain an operation video sequence; Applying the OpenPose skeleton detection algorithm to the operation video sequence, extracting the coordinates of the operator's key joints, and constructing a human posture skeleton model; Inputting the coordinate sequence of the human posture skeleton model into the spatiotemporal graph convolutional network, processing it through the graph convolution layer and the temporal convolution layer to identify the operator's action type; Extracting standard operation skeleton sequence parameters corresponding to the hardware accessories from a safe operation database as a safe operation reference standard; Performing a dynamic time warping algorithm calculation on the operator's action type and the safety operation reference standard to obtain a time-aligned action sequence; The Euclidean distance between the time-aligned action sequence and the safe operation reference standard in three dimensions: joint angle, action amplitude, and operation sequence is calculated to obtain an operation deviation value.

6. The hardware accessories assembly safety testing method based on artificial intelligence according to claim 1 is characterized in that: Normalizing the data collected by the safety test cell line and the deviation value, and calculating the safety adaptation score of the hardware accessories under the current operating conditions, includes: Performing Min-Max normalization processing on the hardware accessories image data collected by the safety test cell line, and if the image pixel value exceeds a preset threshold, truncating the excess portion to obtain normalized image data; Performing Z-score normalization on the physical parameter data collected by the safety test cell line; if the normalized value deviates from the mean by more than 3 standard deviations, it is determined to be an outlier and replaced with a boundary value to obtain normalized physical parameter data; The deviation value is weighted according to the importance of the joint. If the joint deviation value is greater than the safety threshold, the weight coefficient of the corresponding joint is increased by 50% to obtain a weighted deviation value; Inputting the normalized image data, the standardized physical parameter data, and the weighted deviation value into a graph neural network to construct a ternary relationship graph, and disconnecting nodes if the correlation degree between nodes is lower than 0.3 to obtain an optimized relationship graph; Execute a message passing algorithm on the optimized relationship graph to calculate the information flow between nodes. If the iteration fails to converge after more than 10 iterations, terminate the algorithm and use the last result to obtain the node feature vector. Based on the node feature vector, the Sigmoid function is applied to map the calculation result to the interval [0,1]. If the result is less than 0.6, it is judged as low safety adaptation. If the result is greater than 0.8, it is judged as high safety adaptation, and the safety adaptation score of the hardware accessories under the current operating conditions is obtained.

7. The hardware accessories assembly safety testing method based on artificial intelligence according to claim 1 is characterized in that: The step of performing weighted fusion calculation on the data in the security feature database, the deviation value, and the security adaptation score to generate a hardware accessory safety test result includes: Retrieving historical security data of the currently tested hardware accessory from the security feature database, extracting the historical data subset with the highest similarity through cosine similarity calculation, and obtaining a reference security feature; Assigning initial weight coefficients of 0.3, 0.3, and 0.4 to the reference security feature, the deviation value, and the security adaptation score, respectively, constructing a weighted calculation formula to obtain a three-factor fusion model; The calculation results of the three-factor fusion model are compared with the preset safety level threshold, and the test results are divided into four levels: safety, reminder, warning, and danger to obtain the hardware accessories safety test results.

8. An artificial intelligence-based hardware accessories assembly safety testing system, characterized in that: Used to implement the hardware accessories assembly safety testing method based on artificial intelligence according to any one of claims 1 to 7, the hardware accessories assembly safety testing system based on artificial intelligence comprises: Establish a module for scanning and photographing hardware accessories samples, obtaining geometric data and image data, and establishing a hardware accessories security feature database; An extraction module is used to train a deep learning safety assessment model based on the hardware accessories safety feature database, extract and classify the hardware accessories, and obtain a safety status determination result; An acquisition module is used to deploy a safety test cell line on the production line according to the safety assessment model to collect images and physical parameter data of hardware accessories; An analysis module, configured to collect operator action data using the safety test cell line, analyze the action sequence using a feature extraction algorithm, and calculate the deviation value from the safety operation standard; a processing module, configured to normalize the data collected by the safety test cell line and the deviation value, and calculate the safety adaptation score of the hardware accessories under the current operating conditions; The fusion module is used to perform weighted fusion calculation on the data of the security feature database, the deviation value and the safety adaptation score to generate a hardware accessory safety test result.

9. An artificial intelligence-based hardware accessories assembly safety testing device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the method for testing the safety of hardware accessories assembly based on artificial intelligence as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the processor is enabled to execute the hardware accessories assembly safety testing method based on artificial intelligence as claimed in any one of claims 1 to 7.

Images