Ship component assembly detection equipment, system and method based on artificial intelligence

Abstract

The invention discloses ship component assembly detection equipment, system and method based on artificial intelligence. The system comprises a main control module, an image and data display module, a detection item management module, a model algorithm management module and a detection algorithm training platform. The method comprises the following steps: constructing a standard data set, a standard data model and a training data set of ship components, finely adjusting a data model of a to-be-detected ship component, forming a data set of the to-be-detected ship component, constructing a vector database, adjusting and optimizing an artificial intelligence detection algorithm model, and selecting a new detection item of the to-be-detected ship component. And matching the artificial intelligence detection algorithm model, outputting a detection result, and carrying out quality classification on the to-be-detected ship component according to technical index requirements. According to the invention, machine vision detection and a detection algorithm training platform are combined, efficient tuning and reasoning of a vision detection algorithm are realized, the vision detection precision of ship products is improved, the vision detection cost is reduced, and the accuracy of a measurement result is ensured.

Description

Technical Field

[0001] The present invention relates to a ship component assembly detection system and method, and in particular to a ship component assembly detection device, system and method based on artificial intelligence. Background Art

[0002] In the existing ship component assembly process, manual inspection of size or surface defects is generally used. This inspection method usually has the disadvantages of long manual inspection time and low production efficiency. The existing visual inspection is not flexible and flexible, and only supports special weld surface defect detection. The inspection precision and accuracy of the assembly gap and dimensional tolerance of parts are low, which increases quality risks and quality management costs. Although machine vision has been applied to the identification of some weld surface defects, due to the large size of ship component parts and the existence of special-shaped structures, the algorithm model has high requirements, and ordinary visual inspection cannot be applied in practice. In addition, the number of ship component parts is large and the similarity is high, which is repetitive labor. Manual assembly is prone to misassembly and missing assembly. The quality problems in the front assembly process are transferred to the back process, which will cause quality risks, reduce the efficiency of ship construction, increase the cost of rework, and increase the development cycle of ship construction. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a ship component assembly detection equipment based on artificial intelligence with reasonable design and improved visual detection accuracy of ship products in view of the deficiencies in the prior art.

[0004] Another technical problem to be solved by the present invention is to provide an artificial intelligence-based ship component assembly detection system with reasonable design, improved detection efficiency and reduced costs in view of the deficiencies in the prior art.

[0005] Another technical problem to be solved by the present invention is to provide a ship parts inspection method with a reasonable design and using the above-mentioned ship component assembly inspection system based on artificial intelligence in view of the shortcomings of the prior art. The method solves the problems of low efficiency and high production cost of existing manual inspection, as well as the problems of low accuracy, single inspection product type and poor flexibility of existing ordinary visual inspection.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A ship component assembly detection device based on artificial intelligence, characterized in that the device includes a mechanical module, an automatic control module, an image processing module and a computer system module.

[0008] The mechanical module includes an operating machine, a visual inspection module, a sensing module, a linear motion unit and a rotating mechanism, wherein the visual inspection module is used to collect images and graphics of products during the assembly process; the sensing module includes various sensors for controlling the operation of the linear motion unit and the rotating mechanism; the linear motion unit is used to control the movement of products and clamping components during the assembly process; the rotating mechanism is used to control the rotation of products during the assembly process, and cooperate to complete relevant inspection work in accordance with standard requirements;

[0009] The image processing module is used to perform image recognition and matching, image and graphic positioning, size calculation, and analysis and processing on the collected images and graphics;

[0010] The automatic control module is used to assist the information exchange between the visual inspection module and the sensing module to realize the incoming material identification and the inspection result judgment of the product;

[0011] The computer system module is used to realize the detection task reception, detection data statistics, database storage of detection data, human-computer interaction interface, and auxiliary communication of the automatic control module.

[0012] A ship component assembly detection system based on artificial intelligence, characterized in that the system comprises the ship component assembly detection device according to claim 1, and also comprises a main control module, a human-computer interaction module, an equipment control module, a data management module, an image and data display module, a detection project management module, a model algorithm management module and a ship component detection algorithm training platform;

[0013] The main control module is the dispatching hub of the data and commands of the above modules, and has the functions of system startup / shutdown, work report input / output, continuous detection, and detection project setting;

[0014] The human-computer interaction module is used to realize equipment maintenance, testing, and result data management in the whole process of human-computer interaction; it has the function of displaying the current status of the equipment, the current test items, the start or end of continuous testing, the test results during the testing process, and the statistical information of a certain test;

[0015] The equipment control module is responsible for communicating with the electrical system to control the normal operation of the mechanical module according to claim 1;

[0016] The data management module is used to process the data obtained from the visual detection module and the sensing module, and has the functions of data processing and data labeling. It normalizes the collected data, cleans and removes duplicates, and then labels the data;

[0017] The detection project management module is used to load the project configuration file and map the project configuration into the memory for other modules to call, and also receive the order information issued by the production management system;

[0018] The image and data display module uses a visual graphic interface to display the image of the visual imaging unit in real time, the measurement data of the contact measurement part in real time, and the system operation status in real time, so as to timely and accurately reflect the system operation status and various test results;

[0019] The model algorithm management module processes the images collected by the visual inspection module of each inspection station, and obtains the measurement results of various dimensions according to the optimized algorithm operation, including image filtering preprocessing, feature extraction, Sobel edge detection and sub-pixel image segmentation;

[0020] The ship component detection algorithm training platform is used for training and management of artificial intelligence algorithm models for ship component detection, and can perform data preprocessing and annotation, data set construction, algorithm model training and management, and on-demand deployment of cloud-edge-end models.

[0021] The technical problem to be solved by the present invention can also be achieved through the following technical solutions: the ship component detection algorithm training platform includes a ship component data set, a cloud-edge-end computing power module, a training integration framework, and a model algorithm library. The ship component data set includes data acquisition, data processing, data annotation, and data storage modules. The model algorithm library contains industrial mechanism, industrial algorithm, industrial model, and knowledge graph modules. The ship component data set obtains the data of ship components from the ship component assembly detection equipment, ship component design drawings, and ship component historical detection data described in claim 1. The ship component assembly detection algorithm model obtains the data of ship component assembly in the ship component data set through the cloud-edge-end computing power module, and constructs the detection algorithm model of ship component assembly through model selection, model training and fine-tuning, model evaluation, model tuning, and model deployment, and stores it in the model algorithm library for use by the ship component assembly detection system.

[0022] A ship component assembly detection method based on artificial intelligence, characterized in that the method uses the above-mentioned detection system to detect the assembly accuracy and appearance of ship components. The method includes the following steps: first, constructing a standard data set, a standard data model and a training data set for ship components

[0023] Obtain image and graphic data from the ship component assembly design drawings and historical qualified inspection data of the ship components. The data includes geometric dimension data of the ship components and their parts, assembly accuracy data, assembly process standard data (including pre-assembly preparation, personnel requirements, process requirements and process flow, etc.), 3D modeling and digital virtual assembly data, and construct standard images and standard data sets. The standard data sets serve as comparison sets for subsequent ship components to be tested.

[0024] More specifically, the 3D modeling and digital virtual assembly data include solid model geometry information, auxiliary geometry information, machining datums, geometric tolerances, dimensions, views, captures, datum planes, datum points and coordinate systems, engineering notes, standard notes and material requirements.

[0025] The Labelme tool is used to annotate the standard data of the above-mentioned ship components, obtain the vector data and text data of the ship components, form a standard data set of ship components, and divide the standard data set into a standard training set and a standard test set, wherein the standard training set is used to determine the parameters of the model, and the standard test set is used to judge and optimize the effect of the model, and construct the inspection project of ship components;

[0026] Second, fine-tune the data model of the ship components to be tested, form a data set of the ship components to be tested, and build a vector database

[0027] In this phase, DeepSpeed ​​is used as the training framework and FlashAttention is integrated to improve training efficiency;

[0028] The image and graphics of the component to be inspected are collected through the visual inspection module, and the image processing module is used to extract features of the collected images and graphics to form an image feature vector set, and image recognition matching, image and graphic classification are performed; at the same time, the image processing module uses the image area boundary box coordinates to construct position encoding for the collected images and graphics, and completes image and graphic positioning and size calculation; the above-mentioned image feature vector set and position encoding are coupled to obtain image features with spatial information;

[0029] The image feature vector set is specifically formed as follows:

[0030] The image processing module extracts features from the collected product images and graphics to form an image feature vector set Where d is the dimension of a significant region feature, and n is the number of significant regions;

[0031] Furthermore, the feature scale can be increased in the neural network to improve the algorithm model's ability to extract features of product details in the image.

[0032] After forming the image feature vector set, image recognition matching, image and graphic positioning classification are performed, and the position encoding is constructed using the image region bounding box coordinates. Assume that the bounding box coordinates of each region are:

[0033] q = (x1, y1, x2, y2), where x1, y1 are the coordinates of the lower left corner of the image area, and x2, y2 are the coordinates of the upper right corner of the image area. The absolute position of the area coordinates is calculated, and the area coordinates and area are normalized. The formula is as follows:

[0034]

[0035] Among them, w represents the width of the entire image, and h represents the height of the entire image;

[0036] By coupling the image feature vector and the position encoding vector, the image feature with spatial information is obtained as follows:

[0037]

[0038] Among them, relu is the activation function, m1, m2 are coupling parameter matrices, μ1, μ2 are bias vectors, Represents a coupled operation.

[0039] The acquisition of images and graphics of components to be inspected specifically refers to converting the acquired data of ship component assembly into a form suitable for machine learning models. In the data conversion process, a local data extraction service is used to support data in multiple formats by combining a multimodal OCR model.

[0040] Then, the data management module is used to clean, deduplicate and normalize the image features with spatial information.

[0041] The data cleaning and deduplication mentioned above refers to the use of the Simhash algorithm to remove data with high overlap, and at the same time, the use of high-quality open source data ShareGPT to train the regression model to build a quality assessment module and remove low-quality data;

[0042] Then, on the detection algorithm training platform, through the data annotation interface with front-end interaction, the Labelme tool is used to annotate the above-mentioned multiple image features with spatial information, the Recursive Splitter algorithm is used to divide the annotated text into blocks to obtain text data, and the BGE Chinese encoding model is used to vector encode the annotated data to obtain vector data, and the vector data and text data are stored using the vector database milvus to form a fine-tuning special data set for the ship components to be tested;

[0043] The data annotation of the data model of the component to be inspected refers to establishing a data annotation interface with front-end interaction, introducing manual annotation, and saving the results in csv format, forming a high-quality proprietary data set through an algorithm training platform and manual feedback annotation. The product to be inspected is initially positioned by a fixture, and then combined with the positioning binding of the software, the position and direction of the area to be inspected are automatically found, and the area and size that need to be inspected are automatically summarized and found.

[0044] Use the standard training set in step one to train the improved YOLOv5 network model to obtain the artificial intelligence detection algorithm model of ship components, and then input the fine-tuning special data set of the ship components to be tested into the above artificial intelligence detection algorithm model for reinforcement learning fine-tuning through human feedback to improve the model's performance in visual positioning and recognition of ship components, which can effectively improve the quality of the model and the generalization ability in the field of assembly of medium and large ship components, and can significantly improve detection accuracy, reduce training costs, and improve training efficiency. The improved YOLOv5 network model uses the GhostBottleneck module to replace the convolution module in YOLOv5 to reduce the parameters and calculation amount of the model, and improve the running speed. At the same time, the attention mechanism is added to the backbone network to improve the detection accuracy, and the loss function is introduced to improve the convergence speed during the model training process;

[0045] The introduced loss function is specifically:

[0046]

[0047] Among them, δ is the edge distance parameter, Represents the similarity score of the correctly matched image-text pair, and Represents the similarity score of the hardest negative sample mapping between image and text;

[0048] Third, tuning of AI detection algorithm models

[0049] The fine-tuned AI detection algorithm model in step 2 is evaluated and optimized through the standard test set of ship components. The accuracy of the model is measured by the mean precision, and the tuning is completed when the value is better than the set value. The AI ​​detection algorithm model is tuned using the reward modeling mechanism of human feedback reinforcement learning RLHF. The implementation process of this mechanism includes two parts: reward model training and reinforcement learning. In order to reduce the over-optimization of the reward model and ensure that the reward model can generalize well on unknown data, the KL penalty is introduced.

[0050] Therefore, the human feedback reinforcement learning objective function OP considering the reward model and penalty term is:

[0051]

[0052] in, is the learning strategy model of reinforcement learning RL, π STM is the fine-tuning model, β is the coefficient of the KL penalty term, and Y θ (x, y) refers to the scalar output of the reward model for the multimodal (image and text) x and the response y. w ,y l} is a comparison dataset labeled by human annotators, γ is the pre-training loss coefficient, and γ is 0 when the PPO algorithm model is used.

[0053] During RLHF training, the actor model is responsible for generating responses, the reference model is used to calculate the KL penalty with fixed parameters, the reward model provides the overall reward of the entire response, and the critic model is used to learn the label value of each data; the critic model can first be warmed up for 20-50 initial training steps, and both the critic model and the actor model are updated through the standard PPO algorithm. The number of warm-up steps can be set according to actual conditions, including but not limited to the above 20-50;

[0054] Optimally, accuracy, average precision AP (Average Precision), and mean average precision mAP (mean Average Precision) are used to measure the accuracy of the model, and the number of parameters and detection speed (FPS) are used to measure the calculation amount, size, and inference speed of the model.

[0055] The specific process of tuning is as follows: first, the standard training set obtained in step one is input into the artificial intelligence detection algorithm model for training, and then the fine-tuning special data set of the ship component to be tested in step two is input into the detection algorithm model obtained after the standard training set training in the previous step for fine-tuning, and the random gradient descent algorithm is used to update the network parameters to ensure that the best artificial intelligence detection algorithm model can be obtained after the training is completed. Then, the gradient clipping value, constant learning rate, PPO clipping threshold, KL penalty coefficient β are set and the specified number of trainings are performed, and the trained artificial intelligence detection algorithm model is updated through the standard PPO algorithm, and then the standard test set obtained in step one is used to test and evaluate the performance of the artificial intelligence detection algorithm model, and the tuning of the best artificial intelligence detection algorithm model is completed;

[0056] Fourth, select the new test items of the ship components to be tested and match them with the artificial intelligence algorithm model

[0057] The ship component detection equipment based on artificial intelligence automatically collects the image data of the new ship component to be tested, completes the automatic registration of the image of the ship component to be tested with the standard image described in step 1, thereby selecting a suitable detection item, and obtaining the best artificial intelligence detection algorithm model for the corresponding detection item through matching with the model algorithm management module;

[0058] The detection targets of the items to be detected are part size, assembly gap, upper and lower deviations and appearance; the matching artificial intelligence algorithm model for detection refers to finding two sidelines of corresponding diameters from the image, then calculating the number of pixels between the two sidelines, and then segmenting the edge image pixels according to sub-pixel technology to obtain more accurate values, and combining the calibration values ​​to obtain the measurement results;

[0059] Fifth, output the test results and classify the quality of the ship components to be tested according to the technical index requirements.

[0060] The ship component inspection equipment based on artificial intelligence uses the optimal artificial intelligence inspection algorithm model in step 4 to automatically complete the inspection of the assembly quality of the new ship component to be inspected, judges whether the component to be inspected is qualified or unqualified according to the ship component assembly accuracy requirements, outputs and stores the final inspection result data of the new component to be inspected, and conducts quality classification of the ship component to be inspected.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] (1) Machine vision inspection combined with deep learning realizes non-contact and non-destructive inspection of products. The product inspection process is completed automatically without human interference, which improves the inspection accuracy and efficiency and realizes intelligent inspection;

[0063] (2) The system has developed powerful dedicated software that communicates with various modules such as cameras, automatically receives data, automatically controls, and automatically processes and analyzes data, reducing tedious manual calculations and greatly improving accuracy and efficiency. The software automatically records and stores test data, making test data paperless and digital, greatly improving product traceability. Through artificial intelligence, deep learning of product appearance is carried out to achieve high-speed and accurate defect classification and identification, which greatly improves the recognition rate compared to traditional machine vision technology.

[0064] (3) The present invention is aimed at the lack of reliable and easy-to-use graphical algorithm tools for existing ship visual inspection. When encountering inaccurate inspection problems, the user needs to shut down the machine and professional engineers need to go to the site to take pictures and debug; or retrain the model for the pictures, which is inefficient. Based on the inspection algorithm training platform, the image to be inspected is quickly annotated with data, and efficient tuning and reasoning of the visual inspection algorithm is achieved, which significantly shortens the development cycle of the inspection algorithm when the product is changed, improves the visual inspection accuracy and inspection efficiency of ship products, reduces the visual inspection cost of ship products, and ensures the accuracy of the measurement results. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a flow chart of the steps of the ship component assembly detection method based on artificial intelligence according to the present invention;

[0066] Figure 2 This is a block diagram of the overall system for ship component assembly detection based on artificial intelligence of the present invention;

[0067] Figure 3 This is a structural block diagram of the ship component assembly detection equipment based on artificial intelligence described in Examples 1 and 2 of the present invention;

[0068] Figure 4 This is a schematic diagram of the results of the ship component gap detection based on artificial intelligence described in Example 2;

[0069] In the figure: 101 - mechanical module, 201 - automatic control module, 301 - image processing module, 401 - computer system module. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0071] Reference Figure 1-3 , a ship component assembly detection device based on artificial intelligence, including a mechanical module 101, an automatic control module 201, an image processing module 301 and a computer system module 401;

[0072] The mechanical module is the main structure of the equipment, including the operating platform, positioning and related brackets, linear motion units and rotating mechanisms, etc. Among them, the bracket is used to fix optical instruments and various sensors, etc.; the linear motion unit is used to control the movement of products and clamping components; the rotating mechanism is used to control the rotation of products, and complete relevant testing work in accordance with standard requirements;

[0073] More specifically, the clamping and positioning of the product needs to be considered when designing the clamping assembly. The linear motion unit drives the product to move when the camera takes a picture, so that the camera can scan and collect images from one end of the product to the other.

[0074] Optimally, considering that higher detection accuracy corresponds to higher pixels of the camera, a linear array camera is selected in combination with the cost-effectiveness of the equipment. That is, to ensure the detection accuracy of the system, a high-precision industrial linear array camera, a telecentric lens and a parallel light source are configured, and it is installed horizontally or in other ways according to the characteristics of the actual detection product.

[0075] When the worker places the corresponding workpiece to be inspected on the bracket, clamps the workpiece and presses the start button, the photoelectric switch triggers the motion mechanism to drive the product to move, and at the same time triggers the camera to start collecting images. After the collection is completed, the artificial intelligence algorithm of the detection system is used for fitting processing to complete the detection of the product's outer diameter, length and other dimensions.

[0076] More specifically, when inspecting the assembly gap of a product, according to the inspection requirements, after the camera collects a cross-sectional image at one position, the rotating mechanism can rotate by a set angle, and the camera collects another cross-sectional image at another position, and collects multiple cross-sectional images in sequence to finally obtain the assembly gap. Multi-section measurement ensures the accuracy of the measurement.

[0077] The automatic control module is used to assist the information exchange between the visual inspection module and the sensor, which can realize the incoming material identification and the detection result judgment of the product, ensure the stable operation of the visual recognition module, and improve the detection accuracy;

[0078] The image processing module is used to perform image recognition and matching, image and graphic positioning, size calculation, analysis and processing on the images and graphics collected by the camera;

[0079] The computer system module is used to realize functions such as test data statistics, database storage of test data, human-computer interaction interface, auxiliary communication, etc.

[0080] An artificial intelligence detection system based on the above-mentioned ship component assembly detection equipment, the system includes the above-mentioned ship component assembly detection equipment, and also includes a main control module, a human-computer interaction module, an equipment control module, a data management module, an image and data display module, a detection project management module, a model algorithm management module and a detection algorithm training platform;

[0081] The main control module is the dispatching hub of data and commands of each module of the system, and has functions such as system startup / shutdown, work report input / output, continuous detection, and detection project setting;

[0082] The human-computer interaction module is used to realize equipment maintenance, testing, result data management, etc. in the whole process of human-computer interaction; it has the functions of displaying the current status of the equipment, the current test items, starting or ending continuous testing, the test results during the test process, and the statistical information of a certain test;

[0083] The device control module is responsible for communicating with the electrical system and controlling the normal operation of the device;

[0084] The data management module is used to process camera and laser sensor data, and has functions such as data processing and data labeling. It normalizes the collected data, cleans and removes duplicates, and then labels the data;

[0085] The inspection project management module is used to load the project configuration file and map the project configuration into the memory for other modules to call, and can also receive the order information issued by the production management system;

[0086] The image and data display module uses a visual graphic interface to display the image of the visual imaging unit in real time, the measurement data of the contact measurement part in real time, and the system operation status in real time, so as to timely and accurately reflect the system operation status and various test results;

[0087] In order to reduce the deformation of the image, the error caused by the viewing angle, the distortion of the ordinary lens, and the uncertainty of the boundary caused by the interference of inappropriate light sources, the camera, telecentric lens, and telecentric parallel light source are used to collect the image of the object to be measured, and then the target area is detected. The algorithm finds two edge lines of corresponding diameters from the image, and then calculates the number of pixels between the two edge lines. The edge image pixels are segmented according to the sub-pixel technology to obtain more accurate values, and the detection results are obtained by combining the calibration values.

[0088] The model algorithm management module processes the images collected by the camera units of each inspection station, and obtains the measurement results of various dimensions according to the optimized algorithm calculation, including image filtering preprocessing, feature extraction, Sobel edge detection and sub-pixel image segmentation;

[0089] The visual inspection equipment captures the image contour of the product in real time, processes and filters it through a special algorithm, suppresses the noise of the target image, fits the edge of the product's shape contour, and then calculates the pixel ratio of the required measurement indicators (such as length, diameter, etc.) in the image after calibration, thereby obtaining the actual size of the product;

[0090] The ship component detection algorithm training platform is used for training and management of artificial intelligence algorithm models for ship component detection, and can perform data preprocessing and annotation, data set construction, algorithm model training and management, and on-demand deployment of cloud-edge-end models.

[0091] The detection algorithm training platform includes a ship component data set, a cloud-edge-end computing power module, a training integration framework, and a model algorithm library. The ship component data set includes data acquisition, data processing, data annotation, and data storage modules. The model algorithm library contains industrial mechanism, industrial algorithm, industrial model, and knowledge graph modules. The ship component data set obtains the data of ship components from the ship component assembly detection equipment, ship component design drawings, and ship component historical detection data described in claim 1. The ship component assembly detection algorithm model obtains the data of ship component assembly in the ship component data set through the cloud-edge-end computing power module, and constructs the detection algorithm model of ship component assembly through model selection, model training and fine-tuning, model evaluation, model tuning, and model deployment, and stores it in the model algorithm library for use by the ship component assembly detection system.

[0092] In Example 1, the detection equipment and system of the present invention are applied to the clearance and assembly size detection between the impeller and the inner wall of the shell of a certain ship supercharger. The length of the assembly is about 1500-4500mm, the impeller diameter is 150-690mm, and the size range of the clearance detection is 0.01mm-3mm. The supercharger can be loaded and unloaded manually, and clamped. The supercharger can also be automatically loaded and unloaded, clamped and transported by AGV and loading and unloading robots. The operator presses the detection button, and the equipment automatically completes the detection process and automatically transmits the recorded data in a non-contact visual detection manner.

[0093] The inspection items of the supercharger to be inspected include part size, assembly gap, upper and lower deviations and appearance;

[0094] The detection system in this embodiment is used for the size detection of ship supercharger assembly, connected to the control network of the production line, and connected to the AI ​​server, industrial computer and detection equipment in the same network for operation;

[0095] The ship supercharger assembly detection method based on the ship component assembly detection equipment and system of the artificial intelligence comprises the following steps:

[0096] First, build a standard data set, standard data model and training data set for ship turbochargers

[0097] The image and graphic data are obtained from the ship supercharger assembly design drawings and the historical qualified inspection data of the ship components. The data includes the geometric size data of the ship supercharger and its components, the assembly accuracy data, the assembly process standard data (including pre-assembly preparation, personnel requirements, process requirements and process flow, etc.), 3D modeling and digital virtual assembly data, and the standard image and standard data set are constructed. The standard data set is used as a comparison set for the subsequent ship superchargers to be tested;

[0098] More specifically, the 3D modeling and digital virtual assembly data include solid model geometry information, auxiliary geometry information, machining datums, geometric tolerances, dimensions, views, captures, datum planes, datum points and coordinate systems, engineering notes, standard notes and material requirements.

[0099] Optimally, the system builds a data model for the supercharger to be inspected, so as to achieve the purpose of positioning and binding the workpiece. The function of positioning and binding is to automatically find the position and direction of the area to be inspected after the workpiece is placed out of place, and automatically summarize and find the area and size to be inspected;

[0100] The standard data of the ship supercharger is labeled using the Labelme tool to obtain the vector data and text data of the ship supercharger to form a standard data set of the ship supercharger, and the standard data set is divided into a standard training set and a standard test set. The standard training set is used to determine the parameters of the model, and the standard test set is used to judge and optimize the effect of the model, so as to construct the inspection project of the ship supercharger.

[0101] Second, fine-tune the data model of the ship supercharger to be tested, form the ship supercharger data set to be tested, and build a vector database

[0102] In this phase, DeepSpeed ​​is used as the training framework and FlashAttention is integrated to improve training efficiency;

[0103] The camera configured in this embodiment has a detection field of view of approximately 30 mm × 30 mm, and a detection accuracy of 0.01 mm;

[0104] The image and graphics of the supercharger to be inspected are collected through the visual inspection module, and the image processing module is used to extract features of the collected images and graphics to form an image feature vector set, and image recognition matching, image and graphic classification are performed; at the same time, the image processing module uses the image area boundary box coordinates to construct position codes for the collected images and graphics, and complete image and graphic positioning and size calculation; the above-mentioned image feature vector set and position coding are coupled to obtain image features with spatial information;

[0105] The image feature vector set is specifically formed as follows:

[0106] The image processing module extracts features from the collected product images and graphics to form an image feature vector set Where d is the dimension of a significant region feature, and n is the number of significant regions;

[0107] Furthermore, the feature scale can be increased in the neural network to improve the algorithm model's ability to extract features of product details in the image.

[0108] After forming the image feature vector set, image recognition matching, image and graphic positioning classification are performed, and the position encoding is constructed using the image region bounding box coordinates. Assume that the bounding box coordinates of each region are:

[0109] q = (x1, y1, x2, y2), where x1, y1 are the coordinates of the lower left corner of the image area, and x2, y2 are the coordinates of the upper right corner of the image area. The absolute position of the area coordinates is calculated, and the area coordinates and area are normalized. The formula is as follows:

[0110]

[0111] Among them, w represents the width of the entire image, and h represents the height of the entire image;

[0112] By coupling the image feature vector and the position encoding vector, the image feature with spatial information is obtained as follows:

[0113]

[0114] Among them, relu is the activation function, m1, m2 are coupling parameter matrices, μ1, μ2 are bias vectors, Represents a coupled operation.

[0115] The acquisition of images and graphics of the supercharger to be detected specifically refers to converting the acquired data of the ship supercharger assembly into a form suitable for the machine learning model, that is, processing the data into a unified Json format. During the data processing, a local data extraction service is used. The data extraction service is written based on Python language and supports multiple formats of data such as PDF, Word, and Html by combining a multimodal OCR model.

[0116] Then, the data management module is used to clean, deduplicate and normalize the image features with spatial information.

[0117] The data cleaning and deduplication mentioned above refers to the use of the Simhash algorithm to remove data with high overlap, and at the same time, the use of high-quality open source data ShareGPT to train the regression model to build a quality assessment module and remove low-quality data;

[0118] Then, on the detection algorithm training platform, through the data annotation interface with front-end interaction, the Labelme tool is used to annotate the above-mentioned multiple image features with spatial information. In the process of building the vector database, the Recursive Splitter algorithm is used to divide the annotated text into blocks to obtain text data, and the BGE Chinese encoding model is used to vector encode the annotated data to obtain vector data. The vector data and text data are stored using the vector database milvus to form a special data set for fine-tuning of the ship supercharger to be tested.

[0119] The data labeling of the data model of the supercharger to be inspected refers to establishing a data labeling interface with front-end interaction, introducing manual labeling, and saving the results in csv format, forming a high-quality proprietary data set through an algorithm training platform and manual feedback labeling. The product to be inspected is initially positioned by a fixture, and then combined with the positioning binding of the software, the position and direction of the area to be inspected are automatically found, and the area and size to be inspected are automatically summarized and found.

[0120] To go further, the role of positioning binding is to automatically find the position and direction of the area to be tested when the supercharger is placed off-center, and automatically summarize and find the area and size that need to be tested;

[0121] In this embodiment, the product is initially positioned by a fixture, and then combined with the positioning binding of the software, the detection positioning and target locking of the product are fully guaranteed;

[0122] The improved YOLOv5 network model is trained using the standard training set in step one to obtain an artificial intelligence detection algorithm model for ship superchargers, and then the fine-tuning special data set for the ship supercharger to be tested is input into the above artificial intelligence detection algorithm model for reinforcement learning fine-tuning through human feedback to improve the model's performance in visual positioning and recognition of ship superchargers, which can effectively improve the quality of the model and the generalization ability in the field of assembly of medium and large components of ships, and can significantly improve detection accuracy, reduce training costs, and improve training efficiency. The improved YOLOv5 network model uses the GhostBottleneck module to replace the convolution module in YOLOv5 to reduce the parameters and calculation amount of the model, and improve the running speed. At the same time, the attention mechanism is added to the backbone network to improve the detection accuracy, and the loss function is introduced to improve the convergence speed during the model training process;

[0123] The introduced loss function is specifically:

[0124]

[0125] Among them, δ is the edge distance parameter, Represents the similarity score of the correctly matched image-text pair, and Represents the similarity score of the hardest negative sample mapping between image and text;

[0126] The size of the supercharger to be tested is determined by the following formula: L = N*e

[0127] Wherein, L is the length of the supercharger to be tested; N is the number of pixels, which can be directly counted from the collected image; e is the pixel mapping distance of the detection device, which can be obtained through calibration;

[0128] Optimally, the size calibration can be calculated by the following method: the length 1 of a standard supercharger is known, an image of the standard supercharger is obtained, and the number of pixels n corresponding to the length 1 is obtained by directly counting the image, so the mapping pixel mapping distance value e=1 / n;

[0129] Third, tuning of AI detection algorithm models

[0130] The fine-tuned AI detection algorithm model in step 2 is evaluated and optimized through the standard test set of ship superchargers. The accuracy of the model is measured by the mean average precision, and the tuning is completed when the value is better than the set value. The AI ​​detection algorithm model is tuned using the reward modeling mechanism of human feedback reinforcement learning RLHF. The implementation process of this mechanism includes two parts: reward model training and reinforcement learning. In order to reduce the over-optimization of the reward model and ensure that the reward model can generalize well on unknown data, the KL penalty is introduced.

[0131] Therefore, the human feedback reinforcement learning objective function OP considering the reward model and penalty term is:

[0132]

[0133] in, is the learning strategy model of reinforcement learning RL, π STM is the fine-tuning model, β is the coefficient of the KL penalty term, and γ θ (x, y) refers to the scalar output of the reward model for the multimodal (image and text) x and the response y. w ,y l} is a comparison dataset labeled by human annotators, γ is the pre-training loss coefficient, and γ is 0 when the PPO algorithm model is used.

[0134] During RLHF training, the actor model is responsible for generating responses, the reference model is used to calculate the KL penalty with fixed parameters, the reward model provides the overall reward of the entire response, and the critic model is used to learn the label value of each data; the critic model can first be warmed up for 20 initial training steps, and both the critic model and the actor model are updated through the standard PPO algorithm;

[0135] Optimally, accuracy, average precision AP (Average Precision), and mean average precision mAP (mean Average Precision) are used to measure the accuracy of the model, and the number of parameters and detection speed (FPS) are used to measure the calculation amount, size, and inference speed of the model.

[0136] The specific process of tuning is as follows: first, the standard training set obtained in step one is input into the artificial intelligence detection algorithm model for training, and then the fine-tuning special data set of the ship supercharger to be tested in step two is input into the detection algorithm model obtained after training with the standard training set in the previous step for fine-tuning, and the network parameters are updated using the stochastic gradient descent algorithm to ensure that the best artificial intelligence detection algorithm model can be obtained after the training is completed. Then, the gradient clipping value, constant learning rate, PPO clipping threshold, KL penalty coefficient β are set and the training is performed for a specified number of times, and the trained artificial intelligence detection algorithm model is updated through the standard PPO algorithm, and then the standard test set obtained in step one is used to test and evaluate the performance of the artificial intelligence detection algorithm model, and the tuning of the best artificial intelligence detection algorithm model is completed;

[0137] Fourth, select the test items of the new ship supercharger to be tested and match the artificial intelligence algorithm model

[0138] The ship component detection equipment based on artificial intelligence automatically collects the image data of the new ship supercharger to be tested, completes the automatic registration of the image of the ship supercharger to be tested with the standard image described in step 1, thereby selecting a suitable detection item, and obtaining the best artificial intelligence detection algorithm model for the corresponding detection item through matching with the model algorithm management module;

[0139] More specifically, the prerequisite for realizing automatic registration of the image of the supercharger to be tested and its standard image is to find the effective geometric features in the image and extract these features to ensure image registration;

[0140] The detection targets of the items to be detected are part size, assembly gap, upper and lower deviations and appearance; the matching artificial intelligence algorithm model for detection refers to finding two sidelines of corresponding diameters from the image, then calculating the number of pixels between the two sidelines, and then segmenting the edge image pixels according to sub-pixel technology to obtain more accurate values, and combining the calibration values ​​to obtain the measurement results;

[0141] Fifth, output the test results and classify the quality of the ship turbocharger to be tested according to the technical index requirements.

[0142] The ship component inspection equipment based on artificial intelligence uses the optimal artificial intelligence inspection algorithm model in step 4 to automatically complete the inspection of the assembly quality of the new ship supercharger to be tested, judges whether the supercharger to be tested is qualified or unqualified according to the assembly accuracy requirements of the ship supercharger, outputs and stores the final inspection result data of the new supercharger to be tested, and performs quality classification of the ship supercharger to be tested.

[0143] Optimally, the testing equipment and system can integrate various measurement sensors and SPC statistical analysis software, sort the tested superchargers according to the existing classification standards, strictly control product quality, and the measurement data can be analyzed by SPC statistics in real time.

[0144] Example 2, reference Figure 4 , Figure 4 The gap measurement results of the ship subassembly using the gap measurement algorithm program of the detection system of the present invention are demonstrated, wherein the x-axis coordinate represents the part assembly gap, and the z-axis coordinate represents the part assembly step difference. From the results, it can be seen that the assembly gap of the subassembly is 0.243mm (the difference between the x-axis starting coordinate and the x-axis ending coordinate of the marked point).

[0145] An embodiment of the detection equipment and system disclosed in the present invention is applied to the detection of assembly gaps and assembly dimensions of ship sub-assemblies. The dimensions of sub-assembly parts do not exceed 2.5m×2.5m×2.5m. The visual inspection equipment is connected to the sub-assembly manufacturing production line. After the sub-assembly to be inspected is automatically loaded, unloaded and clamped by a robot, the visual inspection equipment automatically completes the inspection process in a non-contact detection manner and automatically transmits and records data.

[0146] The inspection items to be set up by the inspection team include assembly gap, size, upper and lower deviations and appearance;

[0147] In this embodiment, the small group performs preliminary positioning through the fixture, and then combines the positioning binding of the software to fully ensure the detection positioning and target locking of the product;

[0148] The detection system in this embodiment is used for the size detection of the ship group assembly, connected to the control network of the production line, and connected to the AI ​​server, industrial computer and detection equipment in the same network for operation;

[0149] The ship component assembly detection device and system based on the artificial intelligence ship component assembly detection device and system comprises the following steps:

[0150] First, build a standard data set, standard data model and training data set for the ship group

[0151] Obtain image and graphic data from the ship subassembly design drawings and historical qualified inspection data of ship components. The data includes geometric dimension data of the ship subassembly and its components, assembly accuracy data, assembly process standard data (including pre-assembly preparation, personnel requirements, process requirements and process flow, etc.), 3D modeling and digital virtual assembly data, and construct standard images and standard data sets. The standard data sets serve as comparison sets for subsequent ship subassemblies to be tested.

[0152] More specifically, the 3D modeling and digital virtual assembly data include small solid model geometry information, auxiliary geometry information, machining datums, geometric tolerances, dimensions, views, captures, datum planes, datum points and coordinate systems, engineering notes, standard notes and material requirements.

[0153] Use the Labelme tool to annotate the standard data of the above-mentioned ship group structure, obtain the vector data and text data of the ship group structure, form a standard data set of the ship group structure, and divide the standard data set into a standard training set and a standard test set, wherein the standard training set is used to determine the parameters of the model, and the standard test set is used to judge and optimize the effect of the model, and construct the detection project of the ship group structure;

[0154] Second, fine-tune the data model of the ship group to be tested, form a data set of the ship group to be tested, and build a vector database

[0155] In this phase, DeepSpeed ​​is used as the training framework and FlashAttention is integrated to improve training efficiency;

[0156] The camera detection field of view configured in this embodiment is about 50mm×50mm, and the detection accuracy is more than 1 / 3 of the dimensional tolerance;

[0157] The visual inspection module collects the images and graphics of the small group to be inspected, and the image processing module extracts the features of the collected images and graphics to form an image feature vector set, and performs image recognition matching, image and graphic classification; at the same time, the image processing module uses the image area boundary box coordinates to construct position codes for the collected images and graphics, complete image and graphic positioning, and size calculation; the above-mentioned image feature vector set and position coding are coupled to obtain image features with spatial information;

[0158] The image feature vector set is specifically formed as follows:

[0159] The image processing module extracts features from the collected product images and graphics to form an image feature vector set Where d is the dimension of a significant region feature, and n is the number of significant regions;

[0160] Furthermore, the feature scale can be increased in the neural network to improve the algorithm model's ability to extract features of product details in the image.

[0161] After forming the image feature vector set, image recognition matching, image and graphic positioning classification are performed, and the position encoding is constructed using the image region bounding box coordinates. Assume that the bounding box coordinates of each region are:

[0162] q = (x1, y1, x2, y2), where x1, y1 are the coordinates of the lower left corner of the image area, and x2, y2 are the coordinates of the upper right corner of the image area. The absolute position of the area coordinates is calculated, and the area coordinates and area are normalized. The formula is as follows:

[0163]

[0164] Among them, w represents the width of the entire image, and h represents the height of the entire image;

[0165] By coupling the image feature vector and the position encoding vector, the image feature with spatial information is obtained as follows:

[0166]

[0167] Among them, relu is the activation function, m1, m2 are coupling parameter matrices, μ1, μ2 are bias vectors, Represents a coupled operation.

[0168] The said acquisition of images and graphics of the small group to be detected specifically refers to converting the collected data of the ship small group assembly into a form suitable for the machine learning model, that is, processing the data into a unified Json format. In the data processing process, a local data extraction service is used. The data extraction service is written based on the Python language and supports multiple formats of data such as PDF, Word, and Html by combining a multimodal OCR model;

[0169] Then, the data management module is used to clean, deduplicate and normalize the image features with spatial information.

[0170] The data cleaning and deduplication mentioned above refers to the use of the Simhash algorithm to remove data with high overlap, and at the same time, the use of high-quality open source data ShareGPT to train the regression model to build a quality assessment module and remove low-quality data;

[0171] Then, on the detection algorithm training platform, through the data annotation interface with front-end interaction, the Labelme tool is used to annotate the above-mentioned multiple image features with spatial information. In the process of building the vector database, the Recursive Splitter algorithm is used to divide the annotated text into blocks to obtain text data, and the BGE Chinese encoding model is used to vector encode the annotated data to obtain vector data. The vector data and text data are stored using the vector database milvus to form a fine-tuning dedicated data set for the small group of ships to be tested.

[0172] The data annotation of the data model of the inspection group is to establish a data annotation interface with front-end interaction, introduce manual annotation, save the result in csv format, form a high-quality proprietary data set through an algorithm training platform and manual feedback annotation, and the product to be inspected is initially positioned by the fixture, and then combined with the positioning binding of the software, the position and direction of the area to be inspected are automatically found, and the area and size to be inspected are automatically summarized.

[0173] Furthermore, the role of positioning and binding is to automatically find the position and direction of the area to be detected after the workpiece is placed off-center, and automatically summarize and find out the area and size to be detected. In this embodiment, the product is initially positioned by the fixture, and then combined with the positioning and binding of the software, the detection positioning and target locking of the product are fully guaranteed;

[0174] Use the standard training set in step one to train the improved YOLOv5 network model to obtain an artificial intelligence detection algorithm model for the types, positions and assembly gaps of ship subassemblies, and then input the fine-tuning special data set of the ship subassemblies to be tested into the above artificial intelligence detection algorithm model for reinforcement learning fine-tuning through human feedback to improve the model's performance in visual positioning and recognition of ship subassemblies, which can effectively improve the quality of the model and the generalization ability in the field of assembly of medium and large components of ships, and can significantly improve detection accuracy, reduce training costs, and improve training efficiency. The improved YOLOv5 network model uses the GhostBottleneck module to replace the convolution module in YOLOv5 to reduce the parameters and calculation amount of the model, and improve the running speed. At the same time, the attention mechanism is added to the backbone network to improve the detection accuracy, and the loss function is introduced to improve the convergence speed during the model training process;

[0175] The introduced loss function is specifically:

[0176]

[0177] Among them, δ is the edge distance parameter, Represents the similarity score of the correctly matched image-text pair, and Represents the similarity score of the hardest negative sample mapping between image and text;

[0178] The size of the small group to be tested in this embodiment is determined by the following formula: L = N*e

[0179] Where L is the length of the part to be tested; N is the number of pixels, which can be directly counted from the collected image; e is the pixel mapping distance of the detection equipment, which can be obtained through calibration;

[0180] Optimally, the size calibration can be calculated by the following method: the length 1 of a certain standard group is known, an image of the standard group is obtained, and the number of pixels n corresponding to the length 1 is obtained by directly counting the image, so the mapping pixel mapping distance value e=1 / n;

[0181] Third, tuning of AI detection algorithm models

[0182] The AI ​​detection algorithm model of the type, position and assembly gap of the small group to be detected obtained through fine-tuning in step 2 is evaluated and optimized through the standard test set of the small group to be detected. The accuracy of the model is measured by the mean precision, and when the value is better than the set value, it means that the tuning is completed. The reward modeling mechanism of human feedback reinforcement learning RLHF is used to tune the AI ​​detection algorithm model. The implementation process of this mechanism includes two parts: training of the reward model and reinforcement learning. In order to reduce the over-optimization of the reward model and ensure that the reward model can generalize well on unknown data, the KL penalty is introduced.

[0183] Therefore, the human feedback reinforcement learning objective function OP considering the reward model and penalty term is:

[0184]

[0185] in, is the learning strategy model of reinforcement learning RL, π STM is the fine-tuning model, β is the coefficient of the KL penalty term, and γ θ (x, y) refers to the scalar output of the reward model for the multimodal (image and text) x and the response y. w ,y l} is a comparison dataset labeled by human annotators, γ is the pre-training loss coefficient, and γ is 0 when the PPO algorithm model is used.

[0186] During RLHF training, the actor model is responsible for generating responses, the reference model is used to calculate the KL penalty with fixed parameters, the reward model provides the overall reward of the entire response, and the critic model is used to learn the label value of each data; the critic model can first be warmed up for 50 initial training steps, and both the critic model and the actor model are updated through the standard PPO algorithm;

[0187] Optimally, accuracy, average precision AP (Average Precision), and mean average precision mAP (mean Average Precision) are used to measure the accuracy of the model, and the number of parameters and detection speed (FPS) are used to measure the calculation amount, size, and inference speed of the model.

[0188] The specific process of tuning is as follows: first, the standard training set obtained in step one is input into the artificial intelligence detection algorithm model for training, and then the fine-tuning special data set set up by the ship group to be tested in step two is input into the detection algorithm model obtained after training with the standard training set in the previous step for fine-tuning, and the random gradient descent algorithm is used to update the network parameters to ensure that the best artificial intelligence detection algorithm model can be obtained after the training is completed. Then, the gradient clipping value, constant learning rate, PPO clipping threshold, KL penalty coefficient β are set and the specified number of trainings are performed, and the trained artificial intelligence detection algorithm model is updated through the standard PPO algorithm, and then the standard test set obtained in step one is used to test and evaluate the performance of the artificial intelligence detection algorithm model, and the tuning of the best artificial intelligence detection algorithm model is completed;

[0189] Fourth, select the test items for the new ship group to be tested and match the artificial intelligence algorithm model

[0190] The ship panel detection equipment based on artificial intelligence automatically collects the image data of the new ship panel to be tested, completes the automatic registration of the image of the ship panel to be tested with the standard image described in step 1, thereby selecting the appropriate detection item, and obtaining the best artificial intelligence detection algorithm model for the corresponding detection item through the matching of the model algorithm management module;

[0191] More specifically, the prerequisite for realizing automatic registration of the small image to be tested and its standard image is to find the effective geometric features in the image and extract these features to ensure image registration;

[0192] The matching artificial intelligence algorithm model for detection refers to finding two edge lines of corresponding diameters from the image, then calculating the number of pixels between the two edge lines, and then segmenting the edge image pixels according to the sub-pixel technology to obtain more accurate values, and combining the calibration values ​​to obtain the measurement results;

[0193] Fifth, output the test results and classify the quality of the ship group to be tested according to the technical index requirements.

[0194] The ship component inspection equipment based on artificial intelligence adopts the optimal artificial intelligence inspection algorithm model in step 4 to automatically complete the inspection of the assembly quality of the new ship subassembly to be inspected, judges whether the ship subassembly to be inspected is qualified or unqualified according to the assembly accuracy requirements of the ship subassembly, outputs and stores the final inspection result data of the new ship subassembly to be inspected, and conducts quality classification of the ship subassembly to be inspected.

[0195] Optimally, the testing equipment and system can integrate various measurement sensors and SPC statistical analysis software, sort the tested products according to the existing classification standards, strictly control product quality, and the measurement data can be analyzed by SPC statistics in real time.

[0196] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A ship component assembly detection device based on artificial intelligence, characterized in that: The device includes a mechanical module, an automatic control module, an image processing module and a computer system module. The mechanical module includes an operating machine, a visual inspection module, a sensing module, a linear motion unit and a rotating mechanism, wherein the visual inspection module is used to collect images and graphics of parts during the assembly process of ship components; the sensing module includes various sensors for controlling the operation of the linear motion unit and the rotating mechanism; the linear motion unit is used to control the movement of parts and clamping components during the assembly process; the rotating mechanism is used to control the rotation of parts during the assembly process, and cooperate to complete relevant inspection work in accordance with standard requirements; The image processing module is used to perform image recognition and matching, image and graphic positioning, size calculation, and analysis and processing on the collected images and graphics; The automatic control module is used to assist the information exchange between the visual inspection module and the sensing module to realize the incoming material identification and the inspection result judgment of the product; The computer system module is used to realize the detection task reception, detection data statistics, database storage of detection data, human-computer interaction interface, and auxiliary communication of the automatic control module.

2. A ship component assembly detection system based on artificial intelligence, characterized in that: The system comprises the ship component assembly detection equipment according to claim 1, and also comprises a main control module, a human-computer interaction module, an equipment control module, a data management module, an image and data display module, a detection project management module, a model algorithm management module and a detection algorithm training platform; The main control module is the dispatching hub of the data and commands of the above modules, and has the functions of system startup / shutdown, work report input / output, continuous detection, and detection project setting; The human-computer interaction module is used to realize equipment maintenance, testing, and result data management in the whole process of human-computer interaction; it has the function of displaying the current status of the equipment, the current test items, the start or end of continuous testing, the test results during the testing process, and the statistical information of a certain test; The equipment control module is responsible for communicating with the electrical system to control the normal operation of the mechanical module according to claim 1; The data management module is used to process the data obtained from the visual detection module and the sensing module, and has the functions of data processing and data labeling. It normalizes the collected data, cleans and removes duplicates, and then labels the data; The detection project management module is used to load the project configuration file and map the project configuration into the memory for other modules to call, and also receive the order information issued by the production management system; The image and data display module uses a visual graphic interface to display the image of the visual imaging unit in real time, the measurement data of the contact measurement part in real time, and the system operation status in real time, so as to timely and accurately reflect the system operation status and various test results; The model algorithm management module processes the images collected by the visual inspection module of each inspection station, and obtains the measurement results of various dimensions according to the optimized algorithm operation, including image filtering preprocessing, feature extraction, Sobel edge detection and sub-pixel image segmentation; The detection algorithm training platform is used to build an artificial intelligence detection algorithm model for ship component assembly, and to train and manage the artificial intelligence detection algorithm model. At the same time, the artificial intelligence detection algorithm model is used to perform data preprocessing and annotation, data set construction, algorithm model training and management, and cloud-edge-end model on-demand deployment.

3. The artificial intelligence-based ship component assembly detection system according to claim 2, characterized in that: The detection algorithm training platform includes a ship component data set, a cloud-edge-end computing power module, a training integration framework, and a model algorithm library. The ship component data set includes data acquisition, data processing, data labeling, and data storage modules. The model algorithm library contains industrial mechanism, industrial algorithm, industrial model, and knowledge graph modules. The ship component data set obtains the data of ship components from the ship component assembly detection equipment, ship component design drawings, and ship component historical detection data described in claim 1. The ship component assembly detection algorithm model obtains the data of ship component assembly in the ship component data set through the cloud-edge-end computing power module, and constructs the ship component assembly detection algorithm model through model selection, model training and fine-tuning, model evaluation, model tuning, and model deployment in sequence, and stores it in the model algorithm library for use by the ship component assembly detection system.

4. A ship component assembly detection method based on artificial intelligence, characterized in that: The method uses the detection system described in claim 2 or 3 to detect the assembly accuracy and appearance geometric dimensions of parts during the assembly of ship components. The method comprises the following steps: First, build a standard data set, standard data model and training data set for ship components Obtaining standard data of images and graphics from the ship component assembly design drawings and the historical qualified inspection data of the ship components, the standard data including the geometric dimension data, assembly accuracy data, and assembly process standard data of the ship components and their parts; The Labelme tool is used to annotate the standard data of the above-mentioned ship components, obtain the vector data and text data of the ship components, form a standard data set of ship components, and divide the standard data set into a standard training set and a standard test set, wherein the standard training set is used to determine the parameters of the model, and the standard test set is used to judge and optimize the effect of the model, and construct the inspection project of ship components; Second, fine-tune the data model of the ship components to be tested, form a data set of the ship components to be tested, and build a vector database The image and graphics of the component to be inspected are collected through the visual inspection module, and the image processing module is used to extract features of the collected images and graphics to form an image feature vector set, and image recognition matching, image and graphic classification are performed; at the same time, the image processing module uses the image area boundary box coordinates to construct position encoding for the collected images and graphics, and completes image and graphic positioning and size calculation; the above-mentioned image feature vector set and position encoding are coupled to obtain image features with spatial information; Then, the data management module is used to clean, deduplicate and normalize the image features with spatial information. Then, on the detection algorithm training platform, through the data annotation interface with front-end interaction, the Labelme tool is used to annotate the above-mentioned multiple image features with spatial information, the Recursive Splitter algorithm is used to divide the annotated text into blocks to obtain text data, and the BGE Chinese encoding model is used to vector encode the annotated data to obtain vector data, and the vector data and text data are stored using the vector database milvus to form a fine-tuning special data set for the ship components to be tested; Third, tuning of AI detection algorithm models The improved YOLOv5 network model is trained using the standard training set in step 1 to obtain an artificial intelligence detection algorithm model for ship components, and then the fine-tuning special data set of the ship components to be tested is input into the above artificial intelligence detection algorithm model, and reinforcement learning fine-tuning is performed through human feedback to improve the performance of the model in visual positioning and recognition of ship components, effectively improve the quality of the model and the generalization ability in the field of assembly of medium and large ship components, significantly improve detection accuracy, reduce training costs, and improve training efficiency; the improved YOLOv5 network model uses the GhostBottleneck module to replace the convolution module in YOLOv5 to reduce the parameters and calculation amount of the model, and improve the running speed. At the same time, the attention mechanism is added to the backbone network to improve the detection accuracy, and the loss function is introduced to improve the convergence speed during the model training process; Through the standard test set of ship components, the fine-tuned artificial intelligence detection algorithm model is evaluated and optimized using the average precision mean measurement model, and when the value is better than the set value, it means that the tuning is completed; Fourth, select new inspection items for ship components to be tested and match the artificial intelligence detection algorithm model The ship component detection equipment based on artificial intelligence automatically collects new image data of the ship component to be tested, completes the automatic registration of the image of the new ship component to be tested with the standard image described in step 1, thereby selecting a suitable detection item, and obtaining the best artificial intelligence detection algorithm model for the corresponding detection item through matching with the model algorithm management module; Fifth, output the test results and classify the quality of the ship components to be tested according to the technical index requirements. The ship component inspection equipment based on artificial intelligence uses the best artificial intelligence inspection algorithm model in step 4 to automatically complete the inspection of the assembly quality of the new ship component to be inspected, judges whether the component to be inspected is qualified or unqualified according to the ship component assembly accuracy requirements, outputs and stores the final inspection result data of the new component to be inspected, and classifies the quality of the ship component to be inspected.

5. The method for detecting assembly of ship components based on artificial intelligence according to claim 4, characterized in that: The assembly process standard data described in step one of the method includes pre-assembly preparation, personnel requirements, process requirements and process flow; the three-dimensional modeling and digital virtual assembly data include solid model geometry information, auxiliary geometry information, processing datum, geometric tolerance, size, view, capture, datum plane, datum point and coordinate system, engineering annotation, standard annotation and material requirements.

6. The method for detecting assembly of ship components based on artificial intelligence according to claim 4, characterized in that: The images and graphics of the ship components to be inspected collected in step 2 of the method refer to converting the collected data of the ship component assembly into a form suitable for the machine learning model. In the data conversion process, a local data extraction service is used to support data in multiple formats by combining a multimodal OCR model; Use DeepSpeed ​​as the training framework and integrate FlashAttention to improve training efficiency; The image processing module extracts features from the collected product images and graphics to form an image feature vector set Where d is the dimension of a significant region feature, and n is the number of significant regions; The algorithm model’s ability to extract features of product details in images is improved by increasing feature scale in the neural network. After forming the image feature vector set, image recognition matching, image and graphic positioning classification are performed, and the position encoding is constructed using the image region bounding box coordinates. Assume that the bounding box coordinates of each region are: q = (x1, y1, x2, y2), where x1, y1 are the coordinates of the lower left corner of the image region, and x2, y2 are the coordinates of the upper right corner of the image region. The absolute position of the region coordinates is calculated, and the region coordinates and region area are normalized. The formula is as follows: Among them, w represents the width of the entire image, and h represents the height of the entire image; By coupling the image feature vector and the position encoding vector, the image feature with spatial information is obtained as follows: Among them, relu is the activation function, m1, m2 are coupling parameter matrices, μ1, μ2 are bias vectors, Represents a coupled operation.

7. The method for detecting ship component assembly based on artificial intelligence according to claim 4, characterized in that: The third step of the method is that the inspection items of the ship components to be inspected are part size, assembly gap 、 Upper and lower deviations and appearance, therefore, matching the part size detection algorithm model and the appearance detection algorithm model; The size detection algorithm model finds two edge lines of corresponding diameters from the image, then calculates the number of pixels between the two edge lines, and then segments the edge image pixels based on sub-pixel technology to obtain more accurate values, which are then combined with the calibration values ​​to obtain the measurement results. In order to improve the convergence speed of the algorithm model, the loss function is introduced: Among them, δ is the edge distance parameter, Represents the similarity score of the correctly matched image-text pair, and Represents the similarity score of the hardest negative sample mapping between image and text.

8. The method for detecting assembly of ship components based on artificial intelligence according to claim 4, characterized in that: The tuning described in step 3 of this method uses the reward modeling mechanism of human feedback reinforcement learning RLHF to tune the artificial intelligence detection algorithm model. The implementation process of this mechanism includes two parts: reward model training and reinforcement learning. In order to reduce the over-optimization of the reward model and ensure that the reward model can generalize well on unknown data, KL penalty is introduced. The specific process of tuning is: First, the standard training set obtained in step one is input into the artificial intelligence detection algorithm model for training, and then the fine-tuning special data set of the ship components to be tested in step two is input into the artificial intelligence detection algorithm model obtained after training with the standard training set in the previous step for fine-tuning, and the random gradient descent algorithm is used to update the network parameters to ensure that the best artificial intelligence detection algorithm model is obtained after the training is completed. Then, the gradient clipping value, constant learning rate, PPO clipping threshold, KL penalty coefficient β are set and the specified number of trainings are performed, and the trained artificial intelligence detection algorithm model is updated through the standard PPO algorithm, and then the standard test set obtained in step one is used to test and evaluate the performance of the artificial intelligence detection algorithm model to complete the tuning of the best artificial intelligence detection algorithm model.

9. The method for detecting assembly of ship components based on artificial intelligence according to claim 8, characterized in that: In step 3, the human feedback reinforcement learning objective function OP considering the reward model and penalty term is: in, is the learning strategy model of reinforcement learning RL, π STM is the fine-tuning model, β is the coefficient of the KL penalty term, and γ θ (x, y) refers to the scalar output of the reward model for the multimodal (image and text) x and the response y, D = {x, y w , w l } is a comparison dataset labeled by human annotators, γ is the pre-training loss coefficient, and γ is 0 when the PPO algorithm model is used.

Images