A processing flow control method and system based on artificial intelligence
By setting the image acquisition environmental parameters of the standard station and the station to be adjusted in the metal original processing, and performing ambient light source noise reduction processing, the problem of difficult tool wear detection and poor effect of artificial intelligence detection model is solved, and the problem of Higher processing quality and inspection accuracy.
Patent Information
- Application Number
- CN202411230942.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-09-04
AI Technical Summary
The prior art is difficult to detect processing defects caused by tool wear during the processing of metal originals, resulting in unstable processing quality. The artificial intelligence detection model lacks sufficient training samples and differences in noise of ambient light sources affect the detection effect.
By setting the image acquisition environment parameters of the standard station and the station to be adjusted, using a tool with less wear, generating a standard station, performing ambient light source noise reduction processing, and comparing the image differences through grayscale adjustment parameters and artificial intelligence identification model , adjust the processing steps of the station to be adjusted to improve detection accuracy.
Effectively detect and adjust the tool processing steps of each station, improve the processing quality of mechanical parts, reduce tool wear, improve detection accuracy, and reduce the dependence of artificial intelligence detection models on training samples.
Smart Images

Figure CN118778579B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of intelligent manufacturing, and in particular to a processing flow control method and system based on artificial intelligence. Background Art
[0002] With the continuous development of the mechanical field, more and more automation technologies are applied to the production and manufacturing of mechanical equipment. Whether it is the cutting of metal parts, welding between metal parts, transfer of mechanical components, grasping of mechanical components, etc., after the use of automation technology in these links, not only the requirements in terms of precision are met, but also the efficiency is far higher than manual.
[0003] This application mainly focuses on various processing of metal originals, such as cutting, shaving, milling, etc., especially in the field of production and manufacturing of precision mechanical parts. It is necessary to perform multi-tool and multi-step precision processing on metal originals in order to produce qualified mechanical parts. Before producing mechanical parts, the product 3D model structure diagram and metal material data of the mechanical parts will be obtained first, and then the size and specifications of the metal originals corresponding to the metal materials will be determined, and then programming will be performed for the processing of the metal originals. The content of the processing link includes the selection of processing tools, the setting of the metal original speed, the setting of the tool speed, the setting of the displacement, the setting of the processing angle, etc. After the programming setting of the processing link is completed, it is only necessary to place the metal original of the corresponding specifications on the processing table, the original clamping device clamps the metal original, and the processing tool, the original clamping device and the tool control device can process the metal original according to the programming of the processing link.
[0004] However, in the actual processing, programmers may make programming errors, for example: the processing depth of the tool is not set to the target value, and the tool rotation angle is not set to the target value. Such programming errors will cause the processed mechanical parts to be non-compliant. In order to reduce the problem of the above programming errors, pre-processing detection is usually performed. After the programming of the processing link is completed, 3D processing simulation is usually performed. The simulated mechanical parts generated by simulating the metal original after the whole set of processing codes are processed in the computer are tested to see if the specifications of the simulated mechanical parts and the target mechanical parts match. If the specifications do not match, a warning will be issued, so that the programmer can determine the location of the programming code according to the location where the parameters do not match, and then make modifications.
[0005] However, there are some programming problems in the processing links, and it is impossible to use the method of simulating the processing of metal originals for detection. Different types of tools have their own optimal processing ranges, but in the process of continuous use of tools, even for the same type of tools, the optimal processing range of each tool will be different, and defects such as wear are usually subtle, so that the defects caused by the metal originals during processing are also relatively subtle. The normal wear of general tools can also be used normally within a lower processing degree. Tools with low wear can produce good processing effects within a larger processing range, while tools with high wear can only produce good processing effects within a smaller processing range. When there are multiple stations processing metal originals at the same time, the programming of the processing links will be uniformly set to each station, and the tools of each station will be processed in the same order and degree. For steps with a larger degree of processing, tools with a smaller degree of wear can produce better quality. Tools with a larger degree of wear may often produce processing defects. In addition, steps with a larger degree of processing will increase the loss of the tool with a larger degree of wear itself, and the wear condition of each tool that can be used normally cannot be understood through simple image detection.
[0006] At present, in response to the above problems, manual inspection or artificial intelligence detection models are still needed to detect the processing area corresponding to the tool. When it is determined that there are metal original processing defects caused by tool wear, it is necessary to adjust this processing step of the tool at this station separately. For artificial intelligence detection models, mechanical parts are often customized. Even if such mechanical parts are continuously processed, it is difficult to obtain a large number of training samples for a certain processing area. Secondly, the brightness of the ambient light source of each station is different, which makes the image acquisition noise of each station different, which leads to differences between different training samples. The small number of training samples, the large differences in the image acquisition environment of each station, and the subtle wear defects on the metal originals have led to a decrease in the detection effect of the artificial intelligence detection model, and it is impossible to effectively detect the tools of each station, and then it is impossible to adjust this processing step of the station, resulting in unstable processing quality of some stations and reduced processing quality of mechanical parts. Summary of the invention
[0007] The present application discloses a processing flow control method and system based on artificial intelligence, which are used to improve the processing quality of mechanical parts.
[0008] The first aspect of the present application provides a processing flow control method based on artificial intelligence, comprising:
[0009] Input the processing program into the standard station and the station to be adjusted, start the standard station and the station to be adjusted, so that the standard station and the station to be adjusted process the metal original, the wear of each tool installed on the standard station is less than that of the station to be adjusted, and the image acquisition environment parameters of the standard station and the station to be adjusted are set to be the same;
[0010] The first image and the second image of the standard station and the station to be adjusted after the target processing step are respectively acquired at corresponding image acquisition points by using a camera in the acquisition device;
[0011] Determine processing regions corresponding to target processing steps in the first image and the second image respectively, and generate a first region image and a second region image;
[0012] Performing denoising processing on the first region image and the second region image based on the ambient light source noise of the workstation by using the light source brightness of the ambient light source to generate a first denoised image and a second denoised image;
[0013] generating a first grayscale contrast value according to a first grayscale mean value of the first region image and a second grayscale mean value of the first denoised image;
[0014] generating a second grayscale contrast value according to a third grayscale mean value of the second region image and a fourth grayscale mean value of the second denoised image;
[0015] When the actual difference between the first grayscale contrast value and the second grayscale contrast value is greater than the preset difference, a comparison calculation is performed based on the first grayscale contrast value and the second grayscale contrast value to generate an overall adjustment parameter, a grayscale adjustment parameter is generated based on the actual difference calculation, and the overall adjustment parameter and the compensation parameter are determined as the grayscale adjustment parameter;
[0016] Adjusting the pixels on the second denoised image according to the grayscale adjustment parameter;
[0017] Inputting the first denoised image and the adjusted second denoised image into an artificial intelligence recognition model to generate a similarity probability;
[0018] When the similarity probability is lower than the preset probability, the target processing steps of the workstation to be adjusted are adjusted.
[0019] Optionally, respectively determining processing areas corresponding to the target processing steps in the first image and the second image to generate the first area image and the second area image includes:
[0020] Obtaining fixed position information of the metal original;
[0021] Acquire simulated metal original part data corresponding to the target processing step;
[0022] Determine a simulated image from simulated metal original data according to image acquisition points and fixed position information;
[0023] Determine the processing area in the simulation image and generate a binary image;
[0024] A processing region is determined from the first image and the second image according to the binary image, and a first region image and a second region image are generated.
[0025] Optionally, the acquisition device further includes a light source;
[0026] After the processing program is input into the standard station and the station to be adjusted, and the standard station and the station to be adjusted are started, the first image of the standard station and the station to be adjusted after the target processing step and before the second image are respectively acquired at the corresponding image acquisition points by the camera in the acquisition device, and the processing flow control method further includes:
[0027] At the standard workstation and the workstation to be adjusted, use a robotic arm to move the light source to the fill light point according to the corresponding image acquisition point;
[0028] The light sources of the standard workstation and the workstation to be adjusted are started so that the light sources fill light to the processing area of the preset type in the metal original.
[0029] Optionally, performing denoising processing on the first region image and the second region image based on ambient light source noise of the workstation by using the light source brightness of the ambient light source to generate the first denoised image and the second denoised image includes:
[0030] Determine a first area to be tested in the first area image, where the processing depth is greater than a preset depth;
[0031] Determine the center coordinates according to the grayscale of the pixel points in the area to be tested;
[0032] Obtain the brightness of the light source and the average processing depth of the area to be measured, and obtain the variance compensation parameter according to the brightness of the light source and the average processing depth;
[0033] Determine the standard deviation of the area to be measured, and generate a first filter kernel according to the standard deviation, the center coordinates and the variance compensation parameter;
[0034] Determine the second region to be measured, determine the center coordinates, determine the variance compensation parameters, and generate the second filter kernel for the second region image;
[0035] Use the first filter kernel and the second filter kernel to perform noise reduction on the first to-be-detected area and the second to-be-detected area respectively;
[0036] The first region to be measured and the second region to be measured after noise reduction are determined as a first region image and a second region image.
[0037] Optionally, after adjusting the pixels on the second denoised image according to the grayscale adjustment parameter, the first denoised image and the adjusted second denoised image are input into the artificial intelligence recognition model, and before the similarity probability is generated, the processing flow control method further includes:
[0038] Acquire a product defect image, where the product defect image is a photographed image of a metal original having a target processing defect, where the target processing defect is formed when the metal original is processed by an aged and worn tool;
[0039] Use a 3*3 convolution kernel to perform a convolution operation on the second region image to generate processing features;
[0040] Use a 3*3 convolution kernel to perform convolution operation on the product defect image to generate defect features;
[0041] Performing regional pixel weight generation processing on the processed features to generate weight features;
[0042] Adding the processing feature and the weight feature according to the channel to generate a first superposition feature;
[0043] Generate a channel vector for the first superimposed feature through a global average pooling;
[0044] Perform channel compression processing on the channel vector;
[0045] Generate attention weight parameters for each feature channel in the channel vector;
[0046] Channel superposition of the defect feature and the processing feature is performed according to the attention weight parameter to generate a second superposition feature;
[0047] Reconstructing the feature length and width of the second superimposed feature through the first deconvolution module to generate a reconstructed feature;
[0048] Generate an attention matrix for reconstructing features, and set the features whose attention is less than a preset threshold to zero according to the attention matrix;
[0049] The reconstructed features are restored and output through the output module Conv_out to regenerate the second region image.
[0050] The second aspect of the present application provides a processing flow control system based on artificial intelligence, comprising:
[0051] A starting unit is used to input a processing program into a standard station and a station to be adjusted, start the standard station and the station to be adjusted, so that the standard station and the station to be adjusted process the metal original, the wear of each tool installed on the standard station is less than that of the station to be adjusted, and the image acquisition environment parameters of the standard station and the station to be adjusted are set to be the same;
[0052] A first acquisition unit is used to acquire, through a camera in an acquisition device, a first image and a second image of a standard station and a station to be adjusted after a target processing step at corresponding image acquisition points;
[0053] A first generating unit, used to respectively determine the processing areas corresponding to the target processing steps in the first image and the second image, and generate a first area image and a second area image;
[0054] A second generating unit is used to perform noise reduction processing on the first area image and the second area image based on the ambient light source noise of the workstation by using the light source brightness of the ambient light source to generate a first noise-reduced image and a second noise-reduced image;
[0055] A third generating unit, configured to generate a first grayscale contrast value according to a first grayscale mean value of the first region image and a second grayscale mean value of the first denoised image;
[0056] A fourth generating unit, configured to generate a second grayscale contrast value according to a third grayscale mean value of the second region image and a fourth grayscale mean value of the second denoised image;
[0057] A fifth generating unit, configured to generate an overall adjustment parameter by performing a comparison calculation based on the first grayscale contrast value and the second grayscale contrast value when an actual difference between the first grayscale contrast value and the second grayscale contrast value is greater than a preset difference, and generate a grayscale adjustment parameter by performing a comparison calculation based on the first grayscale contrast value and the second grayscale contrast value, and determine the overall adjustment parameter and the compensation parameter as the grayscale adjustment parameter;
[0058] An adjustment unit, used for adjusting the pixel points on the second denoised image according to the grayscale adjustment parameter;
[0059] A sixth generating unit, configured to input the first denoised image and the adjusted second denoised image into an artificial intelligence recognition model to generate a similarity probability;
[0060] The seventh adjustment unit is used to adjust the target processing step of the workstation to be adjusted when the similarity probability is lower than the preset probability.
[0061] Optionally, the first generating unit includes:
[0062] Obtaining fixed position information of the metal original;
[0063] Acquire simulated metal original part data corresponding to the target processing step;
[0064] Determine a simulated image from simulated metal original data according to image acquisition points and fixed position information;
[0065] Determine the processing area in the simulation image and generate a binary image;
[0066] A processing region is determined from the first image and the second image according to the binary image, and a first region image and a second region image are generated.
[0067] Optionally, the acquisition device further includes a light source;
[0068] After the start-up unit and before the first acquisition unit, the process control system further includes:
[0069] The displacement unit is used to use a mechanical arm to displace the light source to the fill light point according to the corresponding image acquisition point at the standard station and the station to be adjusted;
[0070] The fill light unit is used to start the light sources of the standard workstation and the workstation to be adjusted so that the light source fills light to the processing area of the preset type in the metal original.
[0071] Optionally, the second generating unit includes:
[0072] Determine a first area to be tested in the first area image, where the processing depth is greater than a preset depth;
[0073] Determine the center coordinates according to the grayscale of the pixel points in the area to be tested;
[0074] Obtain the brightness of the light source and the average processing depth of the area to be measured, and obtain the variance compensation parameter according to the brightness of the light source and the average processing depth;
[0075] Determine the standard deviation of the area to be measured, and generate a first filter kernel according to the standard deviation, the center coordinates and the variance compensation parameter;
[0076] Determine the second region to be measured, determine the center coordinates, determine the variance compensation parameters, and generate the second filter kernel for the second region image;
[0077] Use the first filter kernel and the second filter kernel to perform noise reduction on the first to-be-detected area and the second to-be-detected area respectively;
[0078] The first region to be measured and the second region to be measured after noise reduction are determined as a first region image and a second region image.
[0079] Optionally, after the adjustment unit and before the sixth generation unit, the processing flow control system further includes:
[0080] A second acquisition unit is used to acquire a product defect image, where the product defect image is a photographed image of a metal original having a target processing defect, where the target processing defect is caused by an aged and worn tool processing the metal original;
[0081] a seventh generating unit, configured to perform a convolution operation on the second region image using a 3*3 convolution kernel to generate a processing feature;
[0082] An eighth generating unit, used to perform a convolution operation on the product defect image using a 3*3 convolution kernel to generate defect features;
[0083] A ninth generating unit, used for performing a regional pixel weight generating process on the processed feature to generate a weight feature;
[0084] a tenth generating unit, configured to add the processing feature and the weight feature according to the channel to generate a first superposition feature;
[0085] A superposition unit, for generating a channel vector for a first superposition feature through a global average pooling;
[0086] A compression unit, used for performing channel compression processing on a channel vector;
[0087] An eleventh generating unit, used to generate an attention weight parameter for each feature channel in the channel vector;
[0088] a twelfth generating unit, configured to perform channel superposition on the defect feature and the processing feature according to the attention weight parameter to generate a second superposition feature;
[0089] A thirteenth generating unit is used to reconstruct the feature length and width of the second superimposed feature through the first deconvolution module to generate a reconstructed feature;
[0090] A zeroing unit, used to generate an attention matrix for reconstructing features, and to set features whose attention is less than a preset threshold to zero according to the attention matrix;
[0091] The output unit is used to restore and output the reconstructed features through the output module Conv_out to regenerate the second region image.
[0092] A third aspect of the present application provides an electronic device, including:
[0093] Processor, memory, input-output unit, and bus;
[0094] The processor is connected to the memory, the input and output unit, and the bus;
[0095] The memory stores a program, and the processor calls the program to execute the first aspect and any optional intelligent question-answering method of the first aspect.
[0096] The fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, it executes the first aspect and any optional intelligent question-and-answer method of the first aspect.
[0097] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:
[0098] In the present application, the processing program is first input into the standard station and the station to be adjusted, so that the instructions received by the standard station and the station to be adjusted are the same, and then the standard station and the station to be adjusted are started, so that the standard station and the station to be adjusted process the metal original. Among them, the wear of each tool installed on the standard station is less than that of the station to be adjusted, that is, a station with relatively good processing effect is set separately, and the image acquisition environment parameters of the standard station and the station to be adjusted are set the same, such as camera model, shooting angle, aperture, focus distance, etc. The first image and the second image of the standard station and the station to be adjusted after the target processing step are respectively obtained by the camera in the acquisition device at the corresponding image acquisition point. At the image acquisition point of each station, the camera shoots the metal original at a preset angle, and the two images collected are roughly the same. Next, the processing area corresponding to the target processing step in the first image and the second image is determined respectively, and the first area image and the second area image are generated. The first area image and the second area image are subjected to denoising based on the ambient light source noise of the station through the light source brightness of the ambient light source, and the first denoised image and the second denoised image are generated to reduce the interference generated by each station. A first grayscale contrast value is generated according to the first grayscale mean of the first region image and the second grayscale mean of the first denoised image. The first grayscale contrast value can characterize the degree of influence of the light source environment of the first region image. A second grayscale contrast value is generated according to the third grayscale mean of the second region image and the fourth grayscale mean of the second denoised image. The second grayscale contrast value can characterize the degree of influence of the light source environment of the second region image. When the actual difference between the first grayscale contrast value and the second grayscale contrast value is greater than the preset difference, it is determined that there is a large difference in the light source environment between the current standard station and the station to be adjusted. It is necessary to perform a comparison calculation based on the first grayscale contrast value and the second grayscale contrast value to generate an overall adjustment parameter, and the grayscale adjustment parameter is generated based on the actual difference calculation. The overall adjustment parameter and the compensation parameter are determined as the grayscale adjustment parameter. The pixel points on the second denoised image are adjusted according to the grayscale adjustment parameter, so that the second denoised image of the station to be adjusted is adjusted to the first denoised image of the standard station. The first denoised image and the adjusted second denoised image are input into the artificial intelligence recognition model, and the two images are compared to see if there is a difference, and a similarity probability is generated. When the similarity probability is lower than the preset probability, the target processing steps of the workstation to be adjusted are adjusted.
[0099] By using a tool with less wear to generate a standard station, operating it simultaneously with the station to be adjusted, acquiring images at the same processing step, and inputting the images on the standard station and the images on the station to be adjusted into the artificial intelligence recognition model for comparison, compared with the use of the artificial intelligence detection model, a large number of training samples in this field are not required. And the images generated at the two stations can also be used as training samples for the artificial intelligence detection model. Secondly, by filtering and reducing noise and generating grayscale contrast values corresponding to the two station images, the difference in the current light source environment of the two can be compared. The difference in the light source environment can be determined by comparison, and then the grayscale adjustment parameters generated by the first grayscale contrast value, the second grayscale contrast value and the actual difference are adjusted to adjust the image, which can eliminate the difference in the light source environment of the two images, improve the accuracy of the recognition comparison of the subsequent artificial intelligence recognition model, and enable effective detection of the tools at each station. When the tool has a defect in a certain processing step, it can be accurately identified, and this processing step is adjusted to improve the processing quality of each station to be adjusted, reduce the wear of the tool, and thus improve the processing quality of mechanical parts. BRIEF DESCRIPTION OF THE DRAWINGS
[0100] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0101] Figure 1 This is a flow chart of an embodiment of a processing flow control method based on artificial intelligence of the present application;
[0102] Figure 2 This is a flow chart of the first stage of another embodiment of the processing flow control method based on artificial intelligence of the present application;
[0103] Figure 3 This is a flow chart of the second stage of another embodiment of the processing flow control method based on artificial intelligence of the present application;
[0104] Figure 4 This is a flowchart of the third stage of another embodiment of the processing flow control method based on artificial intelligence of the present application;
[0105] Figure 5 This is a flowchart of the fourth stage of another embodiment of the processing flow control method based on artificial intelligence of the present application;
[0106] Figure 6 This is a schematic diagram of an embodiment of a processing flow control system based on artificial intelligence of the present application;
[0107] Figure 7This is a schematic diagram of an embodiment of an electronic device of the present application. DETAILED DESCRIPTION
[0108] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0109] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0110] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0111] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.
[0112] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0113] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0114] In the prior art, manual inspection or artificial intelligence detection model is still required to detect the processing area corresponding to the tool. When it is determined that there are metal original processing defects caused by tool wear, it is necessary to adjust this processing step of the tool at this station separately. However, for artificial intelligence detection models, mechanical parts are often customized. Even if such mechanical parts are continuously processed, it is difficult to obtain a large number of training samples for a certain processing area. Secondly, the brightness of the ambient light source of each station is different, which makes the image acquisition noise of each station different, which leads to differences between different training samples. The small number of training samples, the large differences in the image acquisition environment of each station, and the subtle wear defects on the metal originals lead to a decrease in the detection effect of the artificial intelligence detection model, and it is impossible to effectively detect the tools of each station, and then it is impossible to adjust this processing step of the station, resulting in unstable processing quality of some stations and reducing the processing quality of mechanical parts.
[0115] The present application discloses a processing flow control method and system based on artificial intelligence, which are used to improve the processing quality of mechanical parts.
[0116] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0117] The method of the present application can be applied to a server, a device, a terminal or other devices with logic processing capabilities, and the present application does not limit this. For the convenience of description, the following description is made by taking the execution subject as an example of a terminal.
[0118] See also Figure 1 The present application provides an embodiment of a processing flow control method based on artificial intelligence, comprising:
[0119] 101. Input the processing program into the standard station and the station to be adjusted, start the standard station and the station to be adjusted, so that the standard station and the station to be adjusted process the metal original, the wear of each tool installed on the standard station is less than that of the station to be adjusted, and the image acquisition environment parameters of the standard station and the station to be adjusted are set to be the same;
[0120] In this embodiment, the terminal can set a standard workstation with less tool wear in order to form a stable reference system. It can also determine the degree of use of tools in different workstations and use the latest tools in different workstations as the standard. When it is the turn of the tool for processing, the workstation is used as the standard workstation and the other workstations are used as workstations to be adjusted.
[0121] The image acquisition environment parameter settings of the standard workstation and the workstation to be adjusted are the same, which mainly refers to the shooting angle, shooting point, camera model, etc. These parameters should be kept as consistent as possible.
[0122] 102. Acquire, through a camera in an acquisition device, a first image and a second image of the standard station and the station to be adjusted after the target processing step at corresponding image acquisition points;
[0123] In this embodiment, after the terminal starts the standard workstation and the workstation to be adjusted, a processing step is completed. At this time, each workstation suspends processing, fixes the metal original in the same way, and photographs the processing surface of the metal original at the same image acquisition point to obtain the first image and the second image.
[0124] It should be noted that the shooting can be carried out from multiple angles and multiple points, and the images of the two workstations at the same point and angle are associated for subsequent inspection, so as to strive to inspect every surface of the processing area.
[0125] This embodiment is mainly aimed at the first processing of metal originals by each station. It only needs to let each station perform a complete processing once, decompose each processing step, and for some steps of the same type, they can be classified into one category, and the steps with a large degree of processing can be selected for detection. For example: in the steps of processing metal originals, there are 50 steps using tool A, of which 5 are using tool A to dig grooves of a certain depth, in the order of: 5mm, 1mm, 7mm, 5cm and 9cm.
[0126] At this time, the wear of tool A is not within the range that requires replacement, and the processing depth of 1mm is within the absolute effective processing range of tool A, so this step does not need to be tested, and 5mm, 7mm, and 9mm all need to be tested. If the 5mm test process finds that the processing steps need to be adjusted, then the processing steps corresponding to the remaining 7mm and 9mm processing depths also need to be adjusted, and no further testing is required.
[0127] 103. Determine processing regions corresponding to target processing steps in the first image and the second image respectively, and generate a first region image and a second region image;
[0128] After the terminal obtains the first image and the second image, it needs to determine the processing area in the image. Since there is no reference image, the processing area can only be determined by grayscale analysis of the comparison image of the first image and the second image to reduce the amount of calculation.
[0129] 104. Perform filtering and noise reduction processing based on ambient light source noise of the workstation on the first region image and the second region image according to the light source brightness of the ambient light source, so as to generate a first noise reduction image and a second noise reduction image;
[0130] The terminal performs filtering and noise reduction processing on the first area image and the second area image, which mainly processes the noise of the ambient light source noise of the workstation to generate the first noise reduction image and the second noise reduction image. Since the position of each workstation is different during the processing, even if each workstation is in the same processing site, using a unified site light source as the ambient light, there will be differences in the ambient light source due to the irradiation angle or the aging of the site light source. Usually, one or more light sources of the same type are set as ambient light sources for each workstation next to the metal original, and the light source brightness of each light source is consistent. The influence of ambient light source noise caused by the irradiation angle or the aging of the light source is reduced. However, during the processing, the processing area of the metal original may generate uneven heat due to excessive processing. When one or more light sources illuminate the processing area, a high-speed camera is used for shooting. Due to the large variation in the radiation intensity of the high-temperature object, the high-temperature part will be saturated and the other parts will be underexposed, so that the appearance change of the high-temperature part is covered by the strong light of the light source, so that there is new ambient light source noise in the collected image. In this embodiment, by designing a convolution kernel to filter the ambient light source noise caused by the uneven heat generated in the processing area, this type of noise interference can be reduced. The specific construction method and noise reduction method are shown in Example 2.
[0131] 105. Generate a first grayscale contrast value according to a first grayscale mean value of the first region image and a second grayscale mean value of the first denoised image;
[0132] 106. Generate a second grayscale contrast value according to the third grayscale mean value of the second region image and the fourth grayscale mean value of the second denoised image;
[0133] In this embodiment, the terminal uses the first grayscale mean value of the first area image to and the second grayscale mean of the first denoised image Generate the first grayscale contrast value .
[0134]
[0135] In this embodiment, the second grayscale mean of the first denoised image is As the core, the first grayscale mean before noise reduction is and the second grayscale mean Take the absolute value of the difference to generate the noise reduction grayscale change value , and then the denoising grayscale change value and the second grayscale mean Calculate the first grayscale contrast value , the first grayscale contrast value It is used to represent the illumination level received by the first noise reduction image at the workstation.
[0136] Next, the terminal calculates the grayscale mean of the second region image according to the third grayscale mean of the second region image. and the fourth grayscale mean of the second denoised image Generate a second grayscale contrast value , the formula is as follows:
[0137]
[0138] In this embodiment, the second grayscale mean of the first denoised image is As the core, the first grayscale mean before noise reduction is and the second grayscale mean Take the absolute value of the difference to generate the noise reduction grayscale change value , and then the denoising grayscale change value and the second grayscale mean Calculate the first grayscale contrast value , the first grayscale contrast value It is used to represent the illumination level received by the second noise reduction image at the workstation.
[0139] 107. When the actual difference between the first grayscale contrast value and the second grayscale contrast value is greater than the preset difference, a comparison calculation is performed based on the first grayscale contrast value and the second grayscale contrast value to generate an overall adjustment parameter, and a grayscale adjustment parameter is generated based on the actual difference calculation, and the overall adjustment parameter and the compensation parameter are determined as the grayscale adjustment parameter;
[0140] In this embodiment, when the first grayscale contrast value and the second grayscale contrast value are not much different, it means that the lighting environment levels of the standard workstation and the workstation to be adjusted are similar. No additional adjustment is required, and the first denoised image and the second denoised image can be directly input into the artificial intelligence recognition model for comparative detection.
[0141] Otherwise, it is necessary to adjust the grayscale of the second denoised image. In this embodiment, a grayscale adjustment parameter is generated according to the first grayscale contrast value, the second grayscale contrast value and the actual difference. The grayscale adjustment parameter includes an overall adjustment parameter and compensation parameters , the formula is as follows:
[0142]
[0143]
[0144] is the denoised grayscale difference, i.e. the second grayscale mean of the first denoised image and the fourth grayscale mean of the second denoised image The difference.
[0145] 108. Adjust the pixel points on the second denoised image according to the grayscale adjustment parameter;
[0146] The terminal adjusts the parameters according to the grayscale (overall adjustment parameters and compensation parameters ) Adjust the pixels on the second denoised image, the formula is as follows:
[0147]
[0148] in, The coordinates on the second denoised image are Pixels, The coordinates on the second denoised image are Pixels.
[0149] 109. Input the first denoised image and the adjusted second denoised image into an artificial intelligence recognition model to generate a similarity probability;
[0150] In this embodiment, the terminal inputs the first denoised image and the adjusted second denoised image into the artificial intelligence recognition model, and only compares the difference between the first denoised image and the adjusted second denoised image to determine whether there are product defects caused by tool wear, and generates the similarity probability between the first denoised image and the adjusted second denoised image.
[0151] 110. When the similarity probability is lower than the preset probability, the target processing steps of the workstation to be adjusted are adjusted.
[0152] When the similarity probability is lower than the preset probability, the terminal determines the target processing step and adjusts the target processing step of the workstation to be adjusted. The adjustment method mainly includes replacing other tools on the workstation to perform the target processing step, or disassembling the processing step step by step, for example: the processing depth of 7mm is adjusted to 2mm each time, plus 1mm once. This can make the processing quality of the processing step meet the requirements.
[0153] In this embodiment, the processing program is first input into the standard station and the station to be adjusted, so that the instructions received by the standard station and the station to be adjusted are the same, and then the standard station and the station to be adjusted are started, so that the standard station and the station to be adjusted process the metal original. Among them, the wear of each tool installed on the standard station is less than that of the station to be adjusted, that is, a station with relatively good processing effect is set separately, and the image acquisition environment parameters of the standard station and the station to be adjusted are set the same, such as camera model, shooting angle, aperture, focus distance, etc. The first image and the second image of the standard station and the station to be adjusted after the target processing step are respectively obtained by the camera in the acquisition device at the corresponding image acquisition point. At the image acquisition point of each station, the camera shoots the metal original at a preset angle, and the two images collected are roughly the same. Next, the processing area corresponding to the target processing step in the first image and the second image is determined respectively, and the first area image and the second area image are generated. The first area image and the second area image are subjected to denoising based on the ambient light source noise of the station through the light source brightness of the ambient light source, and the first denoised image and the second denoised image are generated to reduce the interference generated by each station. A first grayscale contrast value is generated according to the first grayscale mean of the first region image and the second grayscale mean of the first denoised image. The first grayscale contrast value can characterize the degree of influence of the light source environment on the first region image. A second grayscale contrast value is generated according to the third grayscale mean of the second region image and the fourth grayscale mean of the second denoised image. The second grayscale contrast value can characterize the degree of influence of the light source environment on the second region image. When the actual difference between the first grayscale contrast value and the second grayscale contrast value is greater than the preset difference, it is determined that there is a large difference in the light source environment between the current standard station and the station to be adjusted, and a grayscale adjustment parameter needs to be generated according to the first grayscale contrast value, the second grayscale contrast value and the actual difference. The pixel points on the second denoised image are adjusted according to the grayscale adjustment parameter, so that the second denoised image of the station to be adjusted is adjusted to the first denoised image of the standard station. The first denoised image and the adjusted second denoised image are input into the artificial intelligence recognition model, and the two images are compared to see if there is a difference, and a similarity probability is generated. When the similarity probability is lower than the preset probability, the target processing step of the station to be adjusted is adjusted.
[0154] By using a tool with less wear to generate a standard station, operating it simultaneously with the station to be adjusted, acquiring images at the same processing step, and inputting the images on the standard station and the images on the station to be adjusted into the artificial intelligence recognition model for comparison, compared with the use of the artificial intelligence detection model, a large number of training samples in this field are not required. And the images generated at the two stations can also be used as training samples for the artificial intelligence detection model. Secondly, by filtering and reducing noise and generating grayscale contrast values corresponding to the two station images, the difference in the current light source environment of the two can be compared. The difference in the light source environment can be determined by comparison, and then the grayscale adjustment parameters generated by the first grayscale contrast value, the second grayscale contrast value and the actual difference are adjusted to adjust the image, which can eliminate the difference in the light source environment of the two images, improve the accuracy of the recognition comparison of the subsequent artificial intelligence recognition model, and enable effective detection of the tools at each station. When the tool has a defect in a certain processing step, it can be accurately identified, and this processing step is adjusted to improve the processing quality of each station to be adjusted, reduce the wear of the tool, and thus improve the processing quality of mechanical parts.
[0155] See also Figure 2 , Figure 3 , Figure 4 and Figure 5 The present application provides an embodiment of a processing flow control method based on artificial intelligence, comprising:
[0156] 201. Input the processing program into the standard station and the station to be adjusted, start the standard station and the station to be adjusted, so that the standard station and the station to be adjusted process the metal original, the wear of each tool installed on the standard station is less than that of the station to be adjusted, and the image acquisition environment parameters of the standard station and the station to be adjusted are set to be the same;
[0157] Step 201 in this embodiment is similar to step 101 in the above embodiment and will not be described again here.
[0158] 202. At the standard workstation and the workstation to be adjusted, use a mechanical arm to move the light source to the fill light point according to the corresponding image acquisition point;
[0159] 203. Start the light sources of the standard station and the station to be adjusted so that the light sources provide supplementary light to the processing area of the preset type in the metal original;
[0160] In some processing steps, the processing depth is too great, which makes it impossible for even ambient light to enter, and the camera on the workstation cannot capture the image of the processing area. In this embodiment, the terminal uses a mechanical arm to move the light source to the fill light point according to the corresponding image acquisition point at the standard workstation and the workstation to be adjusted. When this type of processing area exists, the light sources of the standard workstation and the workstation to be adjusted can be started to make the light source fill light to the preset type of processing area in the metal original. At this time, the lighting environment changes from site light to light source light, and the difference in site light source light also becomes the difference in external light source light.
[0161] The processing area that needs to be supplemented with light is determined mainly based on the simulated metal original data. The simulated metal original data can simulate the processing situation in the computer and can accurately obtain the processing degree of each processing and the shape after completion.
[0162] 204. Acquire, through a camera in an acquisition device, at corresponding image acquisition points, a first image and a second image of the standard station and the station to be adjusted after the target processing step respectively;
[0163] Step 204 in this embodiment is similar to step 102 in the above embodiment and will not be described again here.
[0164] 205. Obtaining fixed position information of the metal original;
[0165] 206. Acquire simulated metal original part data corresponding to the target processing step;
[0166] 207. Determine a simulated image from the simulated metal original data according to the image acquisition point and the fixed position information;
[0167] 208. Determine a processing area in the simulated image and generate a binary image;
[0168] 209. Determine a processing region from the first image and the second image according to the binary image, and generate a first region image and a second region image;
[0169] After the target processing step is performed, the clamping devices of the two workstations are required to fix the metal original at the same position to obtain the simulated metal original data corresponding to the target processing step. The simulated metal original data at this time is the simulated 3D metal original after the target processing step. The processing area is determined in the simulated metal original data through the fixed position information, and the area that the image acquisition device can shoot is determined through simulation, so as to determine at least one suitable image acquisition point. If the processing area is blocked, it is necessary to re-fix the position of the metal original and re-determine the image acquisition point. Then the terminal determines the simulated image from the simulated metal original data according to the image acquisition point and the fixed position information, determines the processing area in the simulated image, and generates a binary image. According to the binary image, the processing area is determined from the first image and the second image, and a first area image and a second area image are generated.
[0170] For example: a rectangular metal original is fixed at one end by a clamping device. After being processed by prop A, a rectangular processing groove is formed on one side of the rectangular parallelepiped. At this time, the process stops and the fixed position information of the metal original is obtained. The simulated 3D image is fully projected in the simulated metal original data through the fixed position information of the metal original. However, it is found that in the simulated metal original data, the rectangular processing groove faces downward at this time, and the camera cannot capture the image. At this time, it is necessary to adjust the fixed position information of the metal original (the clamping device rotates 90 degrees) so that it can be captured by the camera from the side. Both workstations fix the metal original in the same position, and generate a suitable image acquisition point for the camera according to the processing area (the side of the metal original, 15cm away from the rectangular processing groove), so that the two cameras and light sources can capture the image of the rectangular processing groove at the same position (the side of the metal original, 15cm away from the rectangular processing groove). At this time, in the simulated metal original data, the simulated image is also determined from the simulated metal original data according to the image acquisition point and fixed position information. The simulated image, the first image and the second image are all 2D images. Next, it is only necessary to determine the position of the rectangular processing groove from the virtual image, and then the processing area can be found from the first image and the second image according to the binary area method.
[0171] 210. Determine a first region to be tested in the first region image, where the processing depth is greater than a preset depth;
[0172] In some processing steps, there is a part of the processing that belongs to the processing area from shallow to deep. For example, when processing a trapezoidal groove, there is a processing area less than 2mm and the maximum depth is 5mm. This area can be determined in the simulation image and from the first area image and the second area image, which reduces the amount of calculation and only detects the parts where defects may occur.
[0173] 211. Determine the center coordinates according to the grayscale of the pixel points in the area to be measured;
[0174] In this embodiment, the terminal determines the center coordinates according to the grayscale of the pixel points in the area to be tested, specifically, the grayscale mean of the area is determined according to the grayscale size, and then the pixel point area greater than the grayscale mean is screened, the relationship between the pixel point areas is detected, and the pixel point relatively close to the center position is determined as the center coordinate. In addition to the above method, there are other selection methods, which are not limited here.
[0175] 212. Obtain the brightness of the light source and the average processing depth of the area to be measured, and obtain a variance compensation parameter according to the brightness of the light source and the average processing depth;
[0176] When there is a scene that requires an external light source or fill light, the terminal will determine a variance compensation parameter by looking up a table based on the actual output brightness of the external light source (light source brightness) and the average processing depth of the area to be tested. The variance compensation parameter is that before the test, the metal original or the same raw material is used to perform multiple depths of processing, and it is illuminated by an external light source, and the image is quickly captured and the standard deviation of the image is calculated. Specifically, a reference brightness is first determined according to the reflective ability of the metal original (the reference brightness with stronger reflective ability will be set smaller), and images of different depths (single processing to this depth, rather than multiple processing to this depth) are collected at the reference brightness, and then the standard variance of each image is calculated, 2mm is used as the initial point, and the depth and standard deviation are recorded.
[0177] Next, the brightness of the external light source is adjusted, and images are collected at different depths and the variance to be corrected is recorded. The variance compensation parameter is the ratio of the standard variance to the variance to be corrected.
[0178] In actual testing, the light source of each workstation outputs a reference fill light brightness. A brightness detector is used to detect the actual output brightness, and then the variance compensation parameters are determined according to the processing depth.
[0179] 213. Determine the standard deviation of the area to be measured, and generate a first filter kernel according to the standard deviation, the center coordinates and the variance compensation parameter;
[0180] The terminal determines the standard deviation of the area to be measured, and generates a first filter kernel according to the standard deviation, the center coordinates and the variance compensation parameter. The formula is as follows:
[0181]
[0182] in, is the center coordinate, is the direct product operation, is the standard deviation, is the variance compensation parameter, As the first filter kernel, by incorporating variance compensation parameters and combining them with a specially designed filter noise reduction function, the image can greatly reduce the ambient light source noise caused by the uneven heat generated in the processing area.
[0183] 214. Determine a second region to be measured, determine center coordinates, determine variance compensation parameters, and generate a second filter kernel for the second region image;
[0184] 215. Use the first filter kernel and the second filter kernel to perform noise reduction on the first to-be-detected area and the second to-be-detected area respectively;
[0185] 216. Determine the first region to be tested and the second region to be tested after noise reduction as a first region image and a second region image;
[0186] Finally, the terminal uses the first filter core and the second filter core to perform noise reduction on the first area to be tested and the second area to be tested respectively, and determines the first area to be tested and the second area to be tested after noise reduction as the first area image and the second area image, and the noise reduction can be performed according to the real-time situation of the light source. If the light source is not used, only ordinary mean filtering and Gaussian filtering are required.
[0187] 217. Generate a first grayscale contrast value according to a first grayscale mean value of the first region image and a second grayscale mean value of the first denoised image;
[0188] 218. Generate a second grayscale contrast value according to the third grayscale mean value of the second region image and the fourth grayscale mean value of the second denoised image;
[0189] 219. When the actual difference between the first grayscale contrast value and the second grayscale contrast value is greater than the preset difference, a comparison calculation is performed based on the first grayscale contrast value and the second grayscale contrast value to generate an overall adjustment parameter, and a grayscale adjustment parameter is generated based on the actual difference calculation, and the overall adjustment parameter and the compensation parameter are determined as the grayscale adjustment parameter;
[0190] 220. Adjust the pixel points on the second denoised image according to the grayscale adjustment parameter;
[0191] Steps 217 to 220 in this embodiment are similar to steps 105 to 108 in the aforementioned embodiment and are not described again here.
[0192] 221. Acquire a product defect image, where the product defect image is a photographed image of a metal original having a target processing defect, where the target processing defect is caused by an aged and worn tool processing the metal original;
[0193] 222. Use a 3*3 convolution kernel to perform a convolution operation on the second region image to generate a processing feature;
[0194] 223. Use a 3*3 convolution kernel to perform convolution operation on the product defect image to generate defect features;
[0195] In this embodiment, the terminal first uses a 3*3 convolution kernel to perform a convolution operation on the second area image to generate a processing feature, and uses a 3*3 convolution kernel to perform a convolution operation on the product defect image to generate a defect feature.
[0196] 224. Performing regional pixel weight generation processing on the processed features to generate weight features;
[0197] Next, the terminal performs regional pixel weight generation processing on the processing feature to generate a weight feature. Specifically, the present embodiment uses a regional pixel attention module RPA to generate regional pixel weights for processing features. The regional pixel attention module RPA includes a BatchNorm-DefConv-ReLU, a BatchNorm-DefConv, a SigMoid function module and a bilinear interpolation module. BatchNorm-DefConv-ReLU, BatchNorm-DefConv, SigMoid function module and bilinear interpolation module are connected in series in sequence. The BatchNorm-DefConv-ReLU layer and the BatchNorm-DefConv layer here both belong to the feature processing layer, and the SigMoid function is a preset function. The regional pixel attention module RPA, as the first attention mechanism, is used to assign a weight to each regional pixel of the processing feature, so that the neural network pays more attention to areas with obvious processing features.
[0198] 225. Add the processing feature and the weight feature according to the channel to generate a first superposition feature;
[0199] 226. Generate a channel vector for the first superposition feature through a global average pool;
[0200] 227. Perform channel compression processing on the channel vector;
[0201] 228. Generate attention weight parameters for each feature channel in the channel vector;
[0202] In this embodiment, the attention weight parameters of each feature are generated mainly through the attention module Attention. The attention mechanism of this module mainly achieves the effect of selecting feature information (defect features) by allocating normalized weights to different feature channels, enhancing certain channels and suppressing other channels.
[0203] In this embodiment, the channel attention module Attention includes a global average pooling layer, a 1*1Conv-ReLU and a Conv-Sigmoid. The operating principle of the Attention module is described in detail below. Specifically, the first superimposed feature first passes through the global average pooling layer (Global Pooling) of the channel attention module Attention to generate a channel vector, then passes through a 1×1 convolution kernel and a ReLU activation function for channel compression, and then passes through a 1×1 convolution kernel and a Sigmoid activation function to output a normalized one-dimensional vector (attention weight parameter) with a dimension equal to the number of input feature channels. This is the attention weight of each feature channel.
[0204] 229. Channel superimpose the defect feature and the processing feature according to the attention weight parameter to generate a second superimposed feature;
[0205] Next, the terminal needs to perform channel superposition on the defect features and processing features according to the attention weight parameters to generate a second superimposed feature, which is a fusion of the features of the two.
[0206] 230. Reconstruct the feature length and width of the second superimposed feature through the first deconvolution module to generate a reconstructed feature;
[0207] The terminal reconstructs the feature length and width of the second superimposed feature through the first deconvolution module to generate a reconstructed feature. The deconvolution module is also called the Conv_t layer. The terminal uses the first deconvolution module to reconstruct the fifth intermediate feature and the first label feature to increase the feature length and width.
[0208] 231. Generate an attention matrix for reconstructing features, and set features whose attention is less than a preset threshold to zero according to the attention matrix;
[0209] In this embodiment, the attention matrix is generated using the attention Dropout module ADO, which includes BatchNorm-2*2DefConv-ReLU and BatchNorm-2*2DefConv-SigMiod. The attention-based Dropout method is different from the random method used in general Dropout. In this embodiment, attention is used to retain more important feature information to improve the performance and generalization of the model.
[0210] The input reconstructed features are put into BatchNorm-2*2DefConv-ReLU for processing, and then its output is input into BatchNorm-2*2DefConv-SigMiod to generate an attention matrix of the same size as the original features. According to the value of the attention matrix, the neurons at the corresponding positions of the original feature matrix whose attention is less than the first preset threshold are set to zero to reduce the interference of irrelevant pixels.
[0211] 232. Restore and output the reconstructed features through the output module Conv_out to regenerate the second region image;
[0212] Finally, the terminal restores the reconstructed features to the original image size using a convolutional neural output module (Conv_out).
[0213] 233. Input the first denoised image and the adjusted second denoised image into an artificial intelligence recognition model to generate a similarity probability;
[0214] 234. When the similarity probability is lower than the preset probability, the target processing steps of the workstation to be adjusted are adjusted.
[0215] Steps 233 to 234 in this embodiment are similar to steps 109 to 110 in the aforementioned embodiment and are not described again here.
[0216] In this embodiment, the processing program is first input into the standard station and the station to be adjusted, so that the instructions received by the standard station and the station to be adjusted are the same, and then the standard station and the station to be adjusted are started, so that the standard station and the station to be adjusted process the metal original. Among them, the wear of each tool installed on the standard station is less than that of the station to be adjusted, that is, a station with relatively good processing effect is set separately, and the image acquisition environment parameters of the standard station and the station to be adjusted are set the same, such as camera model, shooting angle, aperture, focus distance, etc.
[0217] At the standard station and the station to be adjusted, the robot arm is used to move the light source to the fill light point according to the corresponding image acquisition point. The light sources of the standard station and the station to be adjusted are started so that the light source fills light to the processing area of the preset type in the metal original.
[0218] The camera in the acquisition device obtains the first image and the second image of the standard station and the station to be adjusted after the target processing step at the corresponding image acquisition points. At the image acquisition point of each station, the camera shoots the metal original at a preset angle, and the two images collected are roughly the same.
[0219] Next, the fixed position information of the metal original is obtained. The simulated metal original data corresponding to the target processing step is obtained. The simulated image is determined from the simulated metal original data according to the image acquisition point and the fixed position information. The processing area in the simulated image is determined, and a binary image is generated. The processing area is determined from the first image and the second image according to the binary image, and a first area image and a second area image are generated.
[0220] Determine a first area to be tested in the first area image whose processing depth is greater than a preset depth. Determine the center coordinates according to the grayscale of the pixel points in the area to be tested. Obtain the light source brightness and the average processing depth of the area to be tested, and obtain the variance compensation parameter according to the light source brightness and the average processing depth. Determine the standard deviation of the area to be tested, and generate a first filter kernel according to the standard deviation, the center coordinates and the variance compensation parameter;
[0221] The second region to be tested is determined, the center coordinates are determined, the variance compensation parameters are determined, and the second filter kernel is generated for the second region image. The first filter kernel and the second filter kernel are used to reduce noise for the first region to be tested and the second region to be tested respectively. The first region to be tested and the second region to be tested after noise reduction are determined as the first region image and the second region image.
[0222] A first grayscale contrast value is generated according to the first grayscale mean of the first area image and the second grayscale mean of the first denoised image, and the first grayscale contrast value can characterize the degree of influence of the light source environment on the first area image. A second grayscale contrast value is generated according to the third grayscale mean of the second area image and the fourth grayscale mean of the second denoised image, and the second grayscale contrast value can characterize the degree of influence of the light source environment on the second area image. When the actual difference between the first grayscale contrast value and the second grayscale contrast value is greater than the preset difference, it is determined that there is a large difference in the light source environment between the current standard station and the station to be adjusted, and a grayscale adjustment parameter needs to be generated according to the first grayscale contrast value, the second grayscale contrast value and the actual difference. The pixel points on the second denoised image are adjusted according to the grayscale adjustment parameter, so that the second denoised image of the station to be adjusted is adjusted to the first denoised image of the standard station.
[0223] Obtain a product defect image, which is a photographed image of a metal original with a target processing defect, and the target processing defect is formed by processing the metal original with an aging and worn tool. Use a 3*3 convolution kernel to perform a convolution operation on the second region image to generate a processing feature. Use a 3*3 convolution kernel to perform a convolution operation on the product defect image to generate a defect feature. Perform a regional pixel weight generation process on the processing feature to generate a weight feature. Add the processing feature and the weight feature according to the channel to generate a first superposition feature. Generate a channel vector for the first superposition feature through a global average pool. Perform channel compression on the channel vector. Generate an attention weight parameter for each feature channel in the channel vector. Superimpose the defect feature and the processing feature through the channel according to the attention weight parameter to generate a second superposition feature. Reconstruct the feature length and width of the second superposition feature through the first deconvolution module to generate a reconstructed feature. Generate an attention matrix for the reconstructed feature, and set the feature whose attention is less than a preset threshold to zero according to the attention matrix. Restore and output the reconstructed feature through the output module Conv_out to regenerate the second region image.
[0224] The first denoised image and the adjusted second denoised image are input into the artificial intelligence recognition model to compare whether there are differences between the two images and generate a similarity probability. When the similarity probability is lower than the preset probability, the target processing step of the workstation to be adjusted is adjusted.
[0225] By using a tool with less wear to generate a standard station, operating it simultaneously with the station to be adjusted, acquiring images at the same processing step, and inputting the images on the standard station and the images on the station to be adjusted into the artificial intelligence recognition model for comparison, compared with the use of the artificial intelligence detection model, a large number of training samples in this field are not required. And the images generated at the two stations can also be used as training samples for the artificial intelligence detection model. Secondly, by filtering and reducing noise and generating grayscale contrast values corresponding to the two station images, the difference in the current light source environment of the two can be compared. The difference in the light source environment can be determined by comparison, and then the grayscale adjustment parameters generated by the first grayscale contrast value, the second grayscale contrast value and the actual difference are adjusted to adjust the image, which can eliminate the difference in the light source environment of the two images, improve the accuracy of the recognition comparison of the subsequent artificial intelligence recognition model, and enable effective detection of the tools at each station. When the tool has a defect in a certain processing step, it can be accurately identified, and this processing step is adjusted to improve the processing quality of each station to be adjusted, reduce the wear of the tool, and thus improve the processing quality of mechanical parts.
[0226] Secondly, by incorporating variance compensation parameters into the construction of the filter kernel and combining it with a specially designed filter noise reduction function, the image can greatly reduce the ambient light noise caused by uneven heat generated in the processing area, thereby improving the accuracy of subsequent image recognition and comparison.
[0227] See also Figure 6 The present application provides an embodiment of a processing flow control system based on artificial intelligence, comprising:
[0228] The start unit 601 is used to input the processing program into the standard station and the station to be adjusted, start the standard station and the station to be adjusted, so that the standard station and the station to be adjusted process the metal original, the wear of each tool installed on the standard station is less than that of the station to be adjusted, and the image acquisition environment parameters of the standard station and the station to be adjusted are set to be the same;
[0229] The displacement unit 602 is used to use a mechanical arm to displace the light source to the fill light point according to the corresponding image acquisition point at the standard station and the station to be adjusted;
[0230] The fill light unit 603 is used to start the light sources of the standard station and the station to be adjusted so that the light sources fill light to the processing area of the preset type in the metal original;
[0231] The first acquisition unit 604 is used to respectively acquire the first image and the second image of the standard station and the station to be adjusted after the target processing step at the corresponding image acquisition points through the camera in the acquisition device;
[0232] A first generating unit 605 is used to determine the processing areas corresponding to the target processing steps in the first image and the second image respectively, and generate a first area image and a second area image;
[0233] Optionally, the first generating unit 605 includes:
[0234] Obtaining fixed position information of the metal original;
[0235] Acquire simulated metal original part data corresponding to the target processing step;
[0236] Determine a simulated image from simulated metal original data according to image acquisition points and fixed position information;
[0237] Determine the processing area in the simulation image and generate a binary image;
[0238] A processing region is determined from the first image and the second image according to the binary image, and a first region image and a second region image are generated.
[0239] The second generating unit 606 is configured to perform noise reduction processing on the first region image and the second region image based on the ambient light source noise of the workstation by using the light source brightness of the ambient light source to generate a first noise-reduced image and a second noise-reduced image;
[0240] Optionally, the second generating unit 606 includes:
[0241] Determine a first area to be tested in the first area image, where the processing depth is greater than a preset depth;
[0242] Determine the center coordinates according to the grayscale of the pixel points in the area to be tested;
[0243] Obtain the brightness of the light source and the average processing depth of the area to be measured, and obtain the variance compensation parameter according to the brightness of the light source and the average processing depth;
[0244] Determine the standard deviation of the area to be measured, and generate a first filter kernel according to the standard deviation, the center coordinates and the variance compensation parameter;
[0245] Determine the second region to be measured, determine the center coordinates, determine the variance compensation parameters, and generate the second filter kernel for the second region image;
[0246] Use the first filter kernel and the second filter kernel to perform noise reduction on the first to-be-detected area and the second to-be-detected area respectively;
[0247] The first region to be measured and the second region to be measured after noise reduction are determined as a first region image and a second region image.
[0248] A third generating unit 607, configured to generate a first grayscale contrast value according to a first grayscale mean value of the first region image and a second grayscale mean value of the first denoised image;
[0249] A fourth generating unit 608, configured to generate a second grayscale contrast value according to the third grayscale mean value of the second region image and the fourth grayscale mean value of the second denoised image;
[0250] A fifth generating unit 609 is configured to generate an overall adjustment parameter by performing a comparison calculation based on the first grayscale contrast value and the second grayscale contrast value when the actual difference between the first grayscale contrast value and the second grayscale contrast value is greater than a preset difference, and generate a grayscale adjustment parameter by performing a comparison calculation based on the first grayscale contrast value and the second grayscale contrast value, and determine the overall adjustment parameter and the compensation parameter as the grayscale adjustment parameter.
[0251] An adjustment unit 610, configured to adjust the pixel points on the second denoised image according to the grayscale adjustment parameter;
[0252] The second acquisition unit 611 is used to acquire a product defect image, where the product defect image is a photographed image of a metal original with a target processing defect, where the target processing defect is caused by an aged and worn tool processing the metal original;
[0253] A seventh generating unit 612 is used to perform a convolution operation on the second region image using a 3*3 convolution kernel to generate a processing feature;
[0254] An eighth generating unit 613 is used to perform a convolution operation on the product defect image using a 3*3 convolution kernel to generate defect features;
[0255] A ninth generating unit 614, configured to perform a regional pixel weight generation process on the processed feature to generate a weight feature;
[0256] A tenth generating unit 615, configured to add the processing feature and the weight feature according to the channel to generate a first superposition feature;
[0257] A superposition unit 616, configured to generate a channel vector for the first superposition feature through a global average pool;
[0258] A compression unit 617, used for performing channel compression processing on the channel vector;
[0259] an eleventh generating unit 618, for generating an attention weight parameter for each feature channel in the channel vector;
[0260] A twelfth generating unit 619, configured to perform channel superposition on the defect feature and the processing feature according to the attention weight parameter to generate a second superposition feature;
[0261] A thirteenth generating unit 620 is used to reconstruct the feature length and width of the second superimposed feature through the first deconvolution module to generate a reconstructed feature;
[0262] A zeroing unit 621 is used to generate an attention matrix for reconstructing features, and to zero features whose attention is less than a preset threshold according to the attention matrix;
[0263] An output unit 622, configured to restore and output the reconstructed features through an output module Conv_out, and regenerate a second region image;
[0264] A sixth generating unit 623, configured to input the first denoised image and the adjusted second denoised image into an artificial intelligence recognition model to generate a similarity probability;
[0265] The seventh adjustment unit 624 is used to adjust the target processing step of the workstation to be adjusted when the similarity probability is lower than the preset probability.
[0266] See also Figure 7 , the present application provides an electronic device, including:
[0267] Processor 701 , memory 702 , input / output unit 703 , and bus 704 .
[0268] The processor 701 is connected to the memory 702 , the input and output unit 703 , and the bus 704 .
[0269] The memory 702 stores a program, and the processor 701 calls the program to execute the following steps: Figure 1 , Figure 2 and Figure 3 , Figure 4 and Figure 5 Process control methods in machining.
[0270] The present application provides a computer-readable storage medium, on which a program is stored, and when the program is executed on a computer, the program performs the following steps: Figure 1 , Figure 2 and Figure 3 , Figure 4 and Figure 5 Process control methods in machining.
[0271] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0272] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0273] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0274] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0275] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.
Claims
1. A processing flow control method based on artificial intelligence, characterized in that: include: Inputting a processing program into a standard station and a station to be adjusted, starting the standard station and the station to be adjusted, so that the standard station and the station to be adjusted process the metal original, the wear of each tool installed on the standard station is less than that of the station to be adjusted, and the image acquisition environment parameters of the standard station and the station to be adjusted are set to be the same; Acquire the first image and the second image of the standard station and the station to be adjusted after the target processing step at the corresponding image acquisition points respectively by using the camera in the acquisition device; Determine processing areas corresponding to the target processing steps in the first image and the second image respectively, and generate a first area image and a second area image; Determining the processing areas corresponding to the target processing steps in the first image and the second image respectively, generating a first area image and a second area image, including: acquiring fixed position information of the metal original; acquiring simulated metal original data corresponding to the target processing step; determining a simulated image from the simulated metal original data according to the image acquisition points and the fixed position information; determining the processing area in the simulated image and generating a binary image; determining the processing area from the first image and the second image according to the binary image, generating a first area image and a second area image; Performing denoising processing on the first region image and the second region image based on ambient light source noise of the workstation by using the light source brightness of the ambient light source to generate a first denoised image and a second denoised image; generating a first grayscale contrast value according to a first grayscale mean value of the first region image and a second grayscale mean value of the first denoised image; generating a second grayscale contrast value according to a third grayscale mean value of the second region image and a fourth grayscale mean value of the second denoised image; When the actual difference between the first grayscale contrast value and the second grayscale contrast value is greater than the preset difference, a comparison calculation is performed based on the first grayscale contrast value and the second grayscale contrast value to generate an overall adjustment parameter, a grayscale adjustment parameter is calculated based on the actual difference, and the overall adjustment parameter and the compensation parameter are determined as the grayscale adjustment parameter; Adjusting the pixels on the second denoised image according to the grayscale adjustment parameter; Inputting the first denoised image and the adjusted second denoised image into an artificial intelligence recognition model to generate a similarity probability; When the similarity probability is lower than a preset probability, the target processing step of the workstation to be adjusted is adjusted.
2. The processing flow control method according to claim 1, characterized in that: The acquisition device also includes a light source; After the processing program is input into the standard station and the station to be adjusted, and the standard station and the station to be adjusted are started, the first image of the standard station and the station to be adjusted after the target processing step and before the second image are respectively acquired at the corresponding image acquisition points by the camera in the acquisition device, and the processing flow control method further includes: At the standard workstation and the workstation to be adjusted, a mechanical arm is used to move the light source to a fill light point according to the corresponding image acquisition point; The light sources of the standard workstation and the workstation to be adjusted are started so that the light sources provide supplementary light to a processing area of a preset type in the metal original.
3. The processing flow control method according to claim 2, characterized in that: The first region image and the second region image are subjected to denoising processing based on ambient light source noise of a workstation by using the light source brightness of an ambient light source to generate a first denoised image and a second denoised image, including: Determine a first area to be tested in the first area image, where the processing depth is greater than a preset depth; Determine the center coordinates according to the grayscale of the pixel points in the area to be tested; Obtaining the brightness of the light source and the average processing depth of the area to be measured, and obtaining a variance compensation parameter according to the brightness of the light source and the average processing depth; Determine the standard deviation of the area to be measured, and generate a first filter kernel according to the standard deviation, the center coordinates and the variance compensation parameter; Determining a second region to be measured, determining center coordinates, determining variance compensation parameters, and generating a second filter kernel for the second region image; Using the first filter kernel and the second filter kernel to perform noise reduction on the first to-be-detected area and the second to-be-detected area respectively; The first region to be tested and the second region to be tested after noise reduction are determined as a first region image and a second region image.
4. The processing flow control method according to any one of claims 1 to 3, characterized in that: After adjusting the pixels on the second denoised image according to the grayscale adjustment parameter, inputting the first denoised image and the adjusted second denoised image into an artificial intelligence recognition model, and before generating a similarity probability, the processing flow control method further includes: Acquire a product defect image, wherein the product defect image is a photographed image of the metal original component having a target processing defect, wherein the target processing defect is formed when the metal original component is processed by an aged and worn tool; Using a 3*3 convolution kernel to perform a convolution operation on the second region image to generate processing features; Using a 3*3 convolution kernel to perform a convolution operation on the product defect image to generate defect features; Performing regional pixel weight generation processing on the processing feature to generate a weight feature; Adding the processing feature and the weight feature according to the channel to generate a first superposition feature; Generating a channel vector for the first superimposed feature through a global average pooling; Performing channel compression processing on the channel vector; Generating an attention weight parameter for each feature channel in the channel vector; Perform channel superposition on the defect feature and the processing feature according to the attention weight parameter to generate a second superposition feature; Reconstructing the feature length and width of the second superimposed feature through a first deconvolution module to generate a reconstructed feature; Generate an attention matrix for the reconstructed features, and set the features whose attention is less than a preset threshold to zero according to the attention matrix; The reconstructed features are restored and output through the output module Conv_out to regenerate the second region image.
5. A processing control system based on artificial intelligence, characterized in that: include: A starting unit, used for inputting a processing program into a standard station and a station to be adjusted, starting the standard station and the station to be adjusted, so that the standard station and the station to be adjusted process the metal original, the wear of each tool installed on the standard station is less than that of the station to be adjusted, and the image acquisition environment parameters of the standard station and the station to be adjusted are set to be the same; A first acquisition unit is used to respectively acquire a first image and a second image of the standard station and the station to be adjusted after a target processing step at corresponding image acquisition points through a camera in an acquisition device; A first generating unit, used to respectively determine the processing areas corresponding to the target processing steps in the first image and the second image, and generate a first area image and a second area image; The first generating unit includes: obtaining the fixed position information of the metal original; obtaining the simulated metal original data corresponding to the target processing step; determining the simulated image from the simulated metal original data according to the image acquisition point and the fixed position information; determining the processing area in the simulated image and generating a binary image; determining the processing area from the first image and the second image according to the binary image, and generating a first area image and a second area image; A second generating unit is used to perform noise reduction processing on the first area image and the second area image based on the ambient light source noise of the workstation by using the light source brightness of the ambient light source to generate a first noise-reduced image and a second noise-reduced image; a third generating unit, configured to generate a first grayscale contrast value according to a first grayscale mean value of the first region image and a second grayscale mean value of the first denoised image; a fourth generating unit, configured to generate a second grayscale contrast value according to a third grayscale mean value of the second region image and a fourth grayscale mean value of the second denoised image; a fifth generating unit, configured to generate an overall adjustment parameter by performing a comparison calculation based on the first grayscale contrast value and the second grayscale contrast value when an actual difference between the first grayscale contrast value and the second grayscale contrast value is greater than a preset difference, generate a grayscale adjustment parameter by performing a comparison calculation based on the actual difference, and determine the overall adjustment parameter and the compensation parameter as the grayscale adjustment parameter; An adjustment unit, configured to adjust the pixel points on the second denoised image according to the grayscale adjustment parameter; A sixth generating unit, configured to input the first denoised image and the adjusted second denoised image into an artificial intelligence recognition model to generate a similarity probability; The seventh adjustment unit is used to adjust the target processing step of the workstation to be adjusted when the similarity probability is lower than a preset probability.
6. The process control system according to claim 5, characterized in that: The acquisition device also includes a light source; After the start unit and before the first acquisition unit, the processing flow control system further includes: A displacement unit, used for using a mechanical arm to displace the light source to a fill light point at each of the standard station and the station to be adjusted according to the corresponding image acquisition point; A fill light unit is used to start the light sources of the standard workstation and the workstation to be adjusted so that the light sources fill light to a processing area of a preset type in the metal original.
7. The process control system according to claim 6, characterized in that: The second generation unit includes: Determine a first area to be tested in the first area image, where the processing depth is greater than a preset depth; Determine the center coordinates according to the grayscale of the pixel points in the area to be tested; Obtaining the brightness of the light source and the average processing depth of the area to be measured, and obtaining a variance compensation parameter according to the brightness of the light source and the average processing depth; Determine the standard deviation of the area to be measured, and generate a first filter kernel according to the standard deviation, the center coordinates and the variance compensation parameter; Determining a second region to be measured, determining center coordinates, determining variance compensation parameters, and generating a second filter kernel for the second region image; Using the first filter kernel and the second filter kernel to perform noise reduction on the first to-be-detected area and the second to-be-detected area respectively; The first region to be tested and the second region to be tested after noise reduction are determined as a first region image and a second region image.
8. The processing control system according to any one of claims 5 to 7, characterized in that: After the adjustment unit and before the sixth generation unit, the processing flow control system further includes: A second acquisition unit is used to acquire a product defect image, wherein the product defect image is a photographed image of the metal original with a target processing defect, wherein the target processing defect is formed when an aged and worn tool is used to process the metal original; a seventh generating unit, configured to perform a convolution operation on the second region image using a 3*3 convolution kernel to generate a processing feature; an eighth generating unit, configured to perform a convolution operation on the product defect image using a 3*3 convolution kernel to generate defect features; A ninth generating unit, configured to perform a regional pixel weight generating process on the processing feature to generate a weight feature; a tenth generating unit, configured to add the processing feature and the weight feature according to the channel to generate a first superposition feature; A superposition unit, configured to generate a channel vector for the first superposition feature through a global average pool; A compression unit, used for performing channel compression processing on the channel vector; an eleventh generating unit, configured to generate an attention weight parameter for each feature channel in the channel vector; a twelfth generating unit, configured to perform channel superposition on the defect feature and the processing feature according to the attention weight parameter to generate a second superposition feature; a thirteenth generating unit, configured to reconstruct the feature length and width of the second superimposed feature through a first deconvolution module to generate a reconstructed feature; A zeroing unit, used to generate an attention matrix for the reconstructed features, and to zero features whose attention is less than a preset threshold according to the attention matrix; The output unit is used to restore and output the reconstructed features through the output module Conv_out to regenerate the second region image.
