Intelligent riveting control method based on machine vision

Abstract

The embodiment of the invention discloses an intelligent riveting control method based on machine vision. The method comprises the steps that a riveting position image, collected by a camera module, corresponding to a currently-grabbed riveting part is obtained; the riveting position image is recognized through a riveting recognition model obtained through training, and riveting position information is obtained; and controlling a riveting machine to rivet the currently grabbed riveting part according to the riveting position information. By implementing the embodiment of the invention, the accuracy of controlling automatic riveting is improved, and the stability and safety in a complex riveting task are enhanced.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to an intelligent riveting control method based on machine vision. Background Art

[0002] Riveting machines are widely used in hardware, electronics, automobiles and other industries. They are an important foundation for the development and production of automation equipment. If you want to realize an automated production line, automatic riveting is a key link. If this link is still semi-automatic, it will still be limited by manual operation and cannot truly realize the automation of the production line. In addition, with the current rising labor costs, the need for machine replacement is becoming more and more urgent. The combination of machine learning technology with machine vision, contact detection, quality traceability technology and riveting technology provides a possibility for further improving the efficiency and accuracy of hole and rivet detection, and is an effective means to achieve efficient riveting. Summary of the invention

[0003] The embodiments of the present application disclose a machine vision-based intelligent riveting control method, device, electronic device and storage medium, which improve the accuracy of controlling automatic riveting and enhance the stability and safety in complex riveting tasks.

[0004] The embodiment of the present application discloses an intelligent riveting control method based on machine vision, which is applied to an electronic device, wherein the electronic device is communicatively connected with a camera module and at least one riveting machine, and the method comprises: Acquire a riveting position image corresponding to the currently captured riveted component captured by the camera module; The riveting position image is recognized by using the trained riveting recognition model to obtain riveting position information; According to the riveting position information, the riveting machine is controlled to rivet the currently grasped riveted component.

[0005] As an optional implementation manner, the riveting recognition model obtained through training recognizes the riveting position image to obtain the riveting position information, including: The riveting position image is optimized by using the riveting recognition model to obtain an optimized riveting position image; The optimized riveting position image is identified to obtain riveting position information.

[0006] As an optional implementation manner, the step of identifying the optimized riveting position image to obtain the riveting position information includes: Determining the position, angle coordinates and offset of the currently captured riveted component according to the optimized riveting position image; The position, angle coordinates and offset of the currently captured riveted component are converted to obtain riveting position information.

[0007] As an optional implementation manner, the electronic device is further communicatively connected to the rotating stage, and the method further comprises: A rivet feeding signal is sent to the rotary table to control the rotary table to divide the rivets currently being conveyed.

[0008] As an optional implementation, after controlling the riveting machine to rivet the currently grasped riveted component according to the riveting position information, the method further includes: Acquire riveting inspection images and riveting surface data; Extracting features from the riveting detection image and the riveting surface data respectively to obtain image feature information and surface feature information; The riveting detection model obtains the riveting detection result according to the image feature information and the surface feature information.

[0009] As an optional implementation manner, obtaining a riveting detection result by using a riveting detection model according to the image feature information and the surface feature information includes: Performing dimensionality reduction processing and extraction processing on the image feature information and the surface feature information to obtain processed image feature information and surface feature information; According to the preset classification conditions satisfied by the processed image feature information and surface feature information, the processed image feature information and surface feature information are classified to obtain the riveting detection result.

[0010] As an optional implementation manner, after obtaining the riveting detection result according to the image feature information and the surface feature information by using the riveting detection model, the method further includes: The riveting detection result is judged, and if the riveting detection result indicates that a fault occurs in the current operation, a corresponding recovery strategy is generated according to the riveting detection result.

[0011] The embodiment of the present application discloses an intelligent riveting control device of machine vision, which is applied to an electronic device, wherein the electronic device is communicatively connected with a camera module and at least one riveting machine, and the device comprises: An image acquisition module, used to acquire a riveting position image corresponding to the currently captured riveted component captured by the camera module; An image recognition module, used to recognize the riveting position image through a trained riveting recognition model to obtain riveting position information; The riveting control module is used to control the riveting machine to rivet the currently grasped riveted component according to the riveting position information.

[0012] An embodiment of the present application discloses an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements any one of the machine vision-based intelligent riveting control methods disclosed in the embodiments of the present application.

[0013] The embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, any one of the intelligent riveting control methods based on machine vision disclosed in the embodiment of the present application is implemented.

[0014] Compared with the related art, the embodiments of the present application have the following beneficial effects: The embodiment of the present application provides an intelligent riveting control method, device, electronic device and storage medium based on machine vision, which obtains the riveting position image corresponding to the currently grasped riveted component collected by the camera module; recognizes the riveting position image through the trained riveting recognition model to obtain the riveting position information; and controls the riveting machine to rivet the currently grasped riveted component according to the riveting position information. In the implementation of the embodiment of the present application, the riveting position image corresponding to the currently grasped riveted component obtained is recognized by the riveting recognition model, and the riveting position information can be automatically determined according to the riveting position image. Moreover, the riveting machine can be directly controlled to rivet the currently grasped riveted component according to the obtained riveting position information, without the need for manual determination of the riveting position, so that the automatic riveting control can be achieved while avoiding the error caused by manual determination of the riveting position, improving the accuracy of automatic riveting control, and enhancing the stability and safety in complex riveting tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 This is an application scenario diagram of an intelligent riveting control method based on machine vision disclosed in an embodiment of the present application; Figure 2 It is a flow chart of an intelligent riveting control method based on machine vision disclosed in an embodiment of the present application; Figure 3 It is a schematic diagram of a process of obtaining riveting position information by recognizing a riveting position image through a riveting recognition model obtained through training disclosed in an embodiment of the present application; Figure 4It is a flow chart of another intelligent riveting control method based on machine vision disclosed in an embodiment of the present application; Figure 5 It is a structural schematic diagram of an intelligent riveting control device based on machine vision disclosed in an embodiment of the present application; Figure 6 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0018] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0019] The embodiments of the present application disclose a machine vision-based intelligent riveting control method, device, electronic device and storage medium, which improve the accuracy of controlling automatic riveting and enhance the stability and safety in complex riveting tasks. The following are detailed descriptions.

[0020] See also Figure 1 , Figure 1 : is an application scenario diagram of an intelligent riveting control method based on machine vision disclosed in an embodiment of the present application. The intelligent riveting control method based on machine vision is applicable to an electronic device 101, and the electronic device 101 includes but is not limited to a computer, a computer, etc., which is not limited here. The electronic device 101 can be communicatively connected with a camera module 104 and at least one riveting machine 102, and the riveting machine 102 and the camera module 104 can be fixed on a workbench, wherein the riveting machine 102 can be connected to the workbench through a slide rail, and the current riveted component 103 can be grabbed on the riveting machine 102. The electronic device 101 can control the riveting machine 102 to rivet the currently grabbed riveted component 103 to the target position, thereby realizing automatic riveting.

[0021] The electronic device can obtain the riveting position image corresponding to the currently grasped riveted component 103 captured by the camera module 104, recognize the riveting position image through the trained riveting recognition model, obtain the riveting position information, and control the riveting machine 102 to rivet the currently grasped riveted component 103 according to the riveting position information.

[0022] Figure 2 : is a flow chart of an intelligent riveting control method based on machine vision disclosed in an embodiment of the present application. Figure 2 The described intelligent riveting control method based on machine vision is applicable to the above electronic devices. Figure 2 As shown, the intelligent riveting control method based on machine vision may include the following steps: Step S202, obtaining a riveting position image corresponding to the currently captured riveted component captured by the camera module.

[0023] In the current technology, by setting functions such as automatic loading and tail material collection on the riveting machine, the corresponding riveting tasks can be completed automatically and relatively stably. However, in the corresponding riveting tasks, the efficiency and accuracy need to be further improved.

[0024] In some embodiments, the electronic device can be connected to the camera module for communication so that the electronic device can receive the riveting position image captured by the camera module. The camera module can be an industrial camera or an optical camera, which is not limited here. The riveting position image can be used to record the riveting machine currently performing the riveting task and the riveted parts currently captured by the riveting machine. The camera module can collect the riveting position images corresponding to each riveting machine in the order in which each riveting machine performs the riveting task, and then send the collected riveting position images to the electronic device in sequence, so that the electronic device obtains the riveting position image collected by the camera module. For example, according to the preset order of riveting tasks, riveting machine A needs to perform the first riveting task, and riveting machine B needs to perform the second riveting task, then the camera module can first collect the riveting position image corresponding to riveting machine A, and then collect the riveting position image corresponding to riveting machine B.

[0025] Step S204: Recognize the riveting position image by using the trained riveting recognition model to obtain riveting position information.

[0026] In some embodiments, the electronic device may preprocess the riveting position image to obtain a preprocessed riveting position image, and recognize the riveting position image through the trained riveting recognition model to obtain riveting position information. The electronic device preprocesses the riveting position image to improve the accuracy of the riveting recognition model in recognizing the preprocessed riveting position image. The riveting position information may be used to describe the position where the riveted component recorded in the corresponding riveting position image needs to be riveted.

[0027] As an optional implementation, the riveting recognition model can be obtained by training a sample riveting position image set, wherein the sample riveting position image set includes a plurality of sample riveting position images and target riveting position information corresponding to each sample riveting position image. The electronic device inputs the sample riveting position image set into the riveting recognition model to be trained, obtains predicted riveting position information according to each sample riveting position image through the riveting recognition model to be trained, and recognizes the riveting recognition model to be trained according to the error between the target riveting position information corresponding to each sample riveting position image and the predicted riveting position information, thereby obtaining a trained riveting recognition model.

[0028] Step S206, controlling the riveting machine to rivet the currently captured riveted component according to the riveting position information.

[0029] In some embodiments, the electronic device may generate a corresponding riveting signal based on the riveting position information obtained, and send the currently generated riveting signal to the riveting machine to control the riveting machine to rivet the currently grasped riveted component. Furthermore, the riveting position image acquired by the electronic device may include a machine identifier corresponding to each riveting machine, wherein the machine identifier can uniquely identify the corresponding riveting machine. The electronic device identifies the target riveting machine that currently grasps the riveted component based on the machine identifier recorded in the riveting position image, and sends the riveting signal to the target riveting machine based on the machine identifier corresponding to the target riveting machine to control the target riveting machine to rivet the currently grasped riveted component.

[0030] In some embodiments, the electronic device is also connected to the rotating table. The electronic device can send a rivet feeding signal to the rotating table to control the rotating table to divide the rivets currently being transmitted. The rivet feeding signal can be used to indicate that rivets need to be added at present, and the rivet feeding signal can include information such as the number of rivets to be added and the transmission speed of the added rivets.

[0031] The electronic device sends a rivet feeding signal to the rotary table to control the rotary table to divide the rivets currently being conveyed, and can automatically add rivets during the riveting task to further improve the efficiency of riveting.

[0032] In an embodiment of the present application, the electronic device obtains a riveting position image corresponding to the currently grasped riveted component captured by the camera module, recognizes the riveting position image through the trained riveting recognition model, obtains riveting position information, and controls the riveting machine to rivet the currently grasped riveted component according to the riveting position information. The electronic device recognizes the riveting position image corresponding to the currently grasped riveted component acquired through the riveting recognition model, and can automatically determine the riveting position information according to the riveting position image, and can directly control the riveting machine to rivet the currently grasped riveted component according to the obtained riveting position information, without the need for manual determination of the riveting position, thereby achieving control of automatic riveting while avoiding errors caused by manual determination of the riveting position, improving the accuracy of controlling automatic riveting, and enhancing stability and safety in complex riveting tasks.

[0033] Figure 3 FIG. 1 is a flow chart of the process of obtaining riveting position information by recognizing riveting position images through the riveting recognition model obtained through training disclosed in the embodiment of the present application. Figure 3 As shown, the step recognizes the riveting position image through the trained riveting recognition model to obtain the riveting position information, and also includes the following steps: Step S302, optimizing the riveting position image by using the riveting recognition model to obtain an optimized riveting position image.

[0034] In some embodiments, the electronic device optimizes the riveting position image through the riveting recognition model, which may be grayscale adjustment, median filtering, corrosion processing, expansion processing, and edge detection of the riveting position image to obtain an optimized riveting position image. Optionally, the electronic device may extract features from the optimized riveting position image to obtain image feature information, and calculate the image feature information to obtain the riveting position information. The image feature information may include the number of pixels, the length of the major axis, the length of the minor axis, and the area, perimeter, height, width, elongation, Euler number, duty cycle, elongation, complexity, eccentricity, convex pixel ratio, and distance pixel ratio of the detected part. The establishment of the hole and riveting model depends on the collected position information, geometric information, shape information, etc.

[0035] Step S304, identifying the optimized riveting position image to obtain riveting position information.

[0036] In some embodiments, the electronic device can calculate the riveting position information based on the image feature information extracted from the optimized riveting position image. Further, the electronic device can perform principal component analysis on the image feature information through the riveting recognition model to reduce the dimension and extract the image special effect information to obtain the processed image feature information, and calculate the riveting position information based on the processed image feature information.

[0037] As an optional implementation, the electronic device can determine the position, angular coordinates and offset of the currently captured riveted component based on the optimized riveting position image; perform data conversion on the position, angular coordinates and offset of the currently captured riveted component to obtain riveting position information. The position, angular coordinates and offset of the currently captured riveted component can be calculated using image feature information extracted from the optimized riveting position image. The electronic device can perform data conversion on the position, angular coordinates and offset of the previously captured riveted component to convert the position, angular coordinates and offset of the previously captured riveted component into data that can be recognized by the riveting machine, thereby obtaining riveting position information.

[0038] The electronic device determines the position, angular coordinates and offset of the currently grasped riveted component based on the optimized riveting position image, and performs data conversion on the position, angular coordinates and offset of the currently grasped riveted component to obtain the riveting position information, which can ensure that the riveting machine can automatically perform the corresponding riveting task according to the collected riveting position image, thereby providing a technical basis for improving the efficiency of riveting.

[0039] In an embodiment of the present application, the electronic device optimizes the riveting position image through a riveting recognition model to obtain an optimized riveting position image, and recognizes the optimized riveting position image to obtain riveting position information, which can improve the accuracy of controlling automatic riveting and enhance stability and safety in complex riveting tasks.

[0040] See also Figure 4 , Figure 4 : is a flow chart of another intelligent riveting control method based on machine vision disclosed in an embodiment of the present application. In one embodiment, the method further includes the following steps: Step S402, obtaining a riveting position image corresponding to the currently captured riveted component captured by the camera module.

[0041] Step S404: Recognize the riveting position image by using the trained riveting recognition model to obtain riveting position information.

[0042] Step S406, controlling the riveting machine to rivet the currently captured riveted component according to the riveting position information.

[0043] The description of steps S402 to S406 may refer to the relevant description of steps S202 to S206 in the above embodiment, which will not be repeated here.

[0044] Step S408, obtaining riveting detection images and riveting surface data.

[0045] In some embodiments, the electronic device can be connected to the probe for communication, wherein the probe can be used to collect riveting surface data of the surface of the riveted component in the riveting task. The electronic device can receive a detection signal sent by the probe before collecting the riveting surface data through the probe, and the electronic device identifies the detection signal to determine whether the probe is in normal contact with the inspected area. If the electronic device detects that the probe is not in normal contact with the inspected area according to the detection signal, it can output a prompt message to notify the workshop staff to adjust the probe position to ensure that the probe is in normal contact with the inspected area accurately.

[0046] The electronic device can obtain the riveting detection image collected by the camera module and the riveting surface data collected by the probe, and the electronic device can control the camera module to collect the riveting detection image and control the probe to collect the riveting surface information at every preset time period.

[0047] Optionally, the electronic device may control the probe to quickly collect multiple riveted surface data multiple times, and obtain average riveted surface data for detection based on an average value of the multiple riveted surface data.

[0048] Step S410: extracting features from the riveting detection image and the riveting surface data to obtain image feature information and surface feature information.

[0049] Step S412, obtaining a riveting detection result according to the image feature information and the surface feature information through a riveting detection model.

[0050] In some embodiments, the electronic device may perform dimensionality reduction and extraction processing on the image feature information and the surface feature information to obtain processed image feature information and surface feature information; and classify the processed image feature information and the surface feature information according to the preset classification conditions satisfied by the processed image feature information and the surface feature information to obtain the riveting detection result. Among them, the preset classification condition may be obtained by training with a sample feature set, and the sample feature set includes a plurality of sample image feature information and surface feature information, and the target detection result corresponding to each sample image feature information and the surface feature information. The electronic device obtains the corresponding predicted detection result according to the classification condition to be trained and each sample image feature information and the surface feature information, and adjusts the classification condition according to the difference between the target detection result and the predicted detection result corresponding to each sample image feature information and the surface feature information, until the difference is less than the preset difference threshold, and then obtains the trained classification condition.

[0051] The electronic device performs dimensionality reduction processing and extraction processing on the image feature information and the surface feature information to obtain processed image feature information and surface feature information, and classifies the processed image feature information and the surface feature information according to preset classification conditions satisfied by the processed image feature information and the surface feature information to obtain the riveting detection result, which can improve the accuracy of the riveting detection result.

[0052] In some embodiments, after obtaining the riveting detection result, the electronic device may judge the riveting detection result, and if the riveting detection result indicates that the current operation has failed, a corresponding recovery strategy is generated according to the riveting detection result. The electronic device may pre-store a recovery strategy corresponding to the riveting detection result indicating that the current operation has failed, so that the electronic device can output the corresponding recovery strategy according to the riveting detection result.

[0053] The electronic device judges the riveting detection result. If the riveting detection result indicates that a fault has occurred in the current operation, a corresponding recovery strategy is generated based on the riveting detection result, which can improve the efficiency of the workshop personnel in solving the faults that occur during riveting, thereby further improving the efficiency of the riveting task.

[0054] In an embodiment of the present application, an electronic device obtains a riveting detection image and riveting surface data, performs feature extraction on the riveting detection image and the riveting surface data respectively, obtains image feature information and surface feature information, obtains a riveting detection result based on the image feature information and the surface feature information through a riveting detection model, and can automatically detect riveting to enhance stability and safety in complex riveting tasks.

[0055] See also Figure 5 , Figure 5 1 is a schematic diagram of the structure of an intelligent riveting control device based on machine vision disclosed in an embodiment of the present application. The device can be applied to the above-mentioned electronic equipment. Figure 5 As shown, the intelligent riveting control device 500 based on machine vision may include: an image acquisition module 501 , an image recognition module 502 and a riveting control module 503 .

[0056] An image acquisition module 501 is used to acquire a riveting position image corresponding to the currently captured riveted component captured by the camera module; An image recognition module 502 is used to recognize the riveting position image through the trained riveting recognition model to obtain riveting position information; The riveting control module 503 is used to control the riveting machine to rivet the currently captured riveted component according to the riveting position information.

[0057] In one embodiment, the image recognition module 502 further includes an image optimization unit and a position recognition unit: An image optimization unit, used for optimizing the riveting position image through the riveting recognition model to obtain an optimized riveting position image; The position recognition unit is used to recognize the optimized riveting position image to obtain the riveting position information.

[0058] In one embodiment, the position recognition unit is further used to determine the position, angle coordinates and offset of the currently grasped riveted component according to the optimized riveting position image; perform data conversion on the position, angle coordinates and offset of the currently grasped riveted component to obtain riveting position information.

[0059] In one embodiment, the intelligent riveting control device 500 based on machine vision further includes a feeding control module: The feeding control module is used to send a rivet feeding signal to the rotary table to control the rotary table to divide the rivets currently being conveyed.

[0060] In one embodiment, the intelligent riveting control device 500 based on machine vision further includes a data acquisition module, a data extraction module and a result detection module: A data acquisition module, used to acquire riveting detection images and riveting surface data; A data extraction module is used to extract features from the riveting detection image and the riveting surface data to obtain image feature information and surface feature information; The result detection module is used to obtain the riveting detection result according to the image feature information and the surface feature information through the riveting detection model.

[0061] In one embodiment, the result detection module is also used to perform dimensionality reduction and extraction processing on the image feature information and the surface feature information to obtain processed image feature information and surface feature information; and classify the processed image feature information and the surface feature information according to preset classification conditions satisfied by the processed image feature information and the surface feature information to obtain the riveting detection result.

[0062] In one embodiment, the intelligent riveting control device 500 based on machine vision further includes a result prompt module: The result prompt module is used to judge the riveting detection result. If the riveting detection result indicates that a fault occurs in the current operation, a corresponding recovery strategy is generated according to the riveting detection result.

[0063] See also Figure 6 , Figure 6 Schematic diagram of the structure of an electronic device disclosed in the embodiment of the present application. Figure 6 As shown, the electronic device 600 may include: A memory 601 storing executable program codes; a processor 602 coupled to the memory 601; The processor 602 calls the executable program code stored in the memory 601 to execute any one of the intelligent riveting control methods based on machine vision disclosed in the embodiments of the present application.

[0064] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by the processor, the processor implements any one of the seat angle adjustment methods disclosed in the embodiment of the present application.

[0065] The embodiments of the present application disclose a computer program product, including a computer program, and the computer program can be executed by a processor to implement the methods described in the above embodiments.

[0066] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required for the present application.

[0067] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the above-mentioned processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0068] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0069] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0070] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, server or network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above-mentioned methods of various embodiments of the present application.

[0071] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0072] The above is a detailed introduction to a machine vision-based intelligent riveting control method, device, electronic device and storage medium disclosed in the embodiment of the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. At the same time, for those of ordinary skill in the art, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. An intelligent riveting control method based on machine vision, characterized in that: Applied to an electronic device, the electronic device is communicatively connected with a camera module and at least one riveting machine, the method comprising: Acquire a riveting position image corresponding to the currently captured riveted component captured by the camera module; The riveting position image is recognized by using the trained riveting recognition model to obtain riveting position information; According to the riveting position information, the riveting machine is controlled to rivet the currently grasped riveted component.

2. The intelligent riveting control method based on machine vision according to claim 1 is characterized in that: The riveting recognition model obtained through training recognizes the riveting position image to obtain riveting position information, including: The riveting position image is optimized by using the riveting recognition model to obtain an optimized riveting position image; The optimized riveting position image is identified to obtain riveting position information.

3. The intelligent riveting control method based on machine vision according to claim 2 is characterized in that: The step of identifying the optimized riveting position image to obtain riveting position information includes: Determining the position, angle coordinates and offset of the currently captured riveted component according to the optimized riveting position image; The position, angle coordinates and offset of the currently captured riveted component are converted to obtain riveting position information.

4. The intelligent riveting control method based on machine vision according to claim 1 is characterized in that: The electronic device is also in communication with the rotating stage, and the method further comprises: A rivet feeding signal is sent to the rotary table to control the rotary table to divide the rivets currently being conveyed.

5. The intelligent riveting control method based on machine vision according to claim 1 is characterized in that: After controlling the riveting machine to rivet the currently grasped riveted component according to the riveting position information, the method further includes: Acquire riveting inspection images and riveting surface data; Extracting features from the riveting detection image and the riveting surface data respectively to obtain image feature information and surface feature information; The riveting detection model obtains the riveting detection result according to the image feature information and the surface feature information.

6. The intelligent riveting control method based on machine vision according to claim 5 is characterized in that: The step of obtaining a riveting detection result according to the image feature information and the surface feature information by using a riveting detection model includes: Performing dimensionality reduction processing and extraction processing on the image feature information and the surface feature information to obtain processed image feature information and surface feature information; According to the preset classification conditions satisfied by the processed image feature information and surface feature information, the processed image feature information and surface feature information are classified to obtain the riveting detection result.

7. The intelligent riveting control method based on machine vision according to claim 5 is characterized in that: After obtaining the riveting detection result according to the image feature information and the surface feature information by the riveting detection model, the method further includes: The riveting detection result is judged, and if the riveting detection result indicates that a fault occurs in the current operation, a corresponding recovery strategy is generated according to the riveting detection result.

8. An intelligent riveting control device based on machine vision, characterized in that: Applied to electronic equipment, the electronic equipment is communicatively connected with a camera module and at least one riveting machine, the device comprises: An image acquisition module, used to acquire a riveting position image corresponding to the currently captured riveted component captured by the camera module; An image recognition module, used to recognize the riveting position image through a trained riveting recognition model to obtain riveting position information; The riveting control module is used to control the riveting machine to rivet the currently grasped riveted component according to the riveting position information.

9. An electronic device, characterized in that: It comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements an intelligent riveting control method based on machine vision as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, an intelligent riveting control method based on machine vision as described in any one of claims 1 to 7 is implemented.

Images