Image processing method and system based on intelligent spraying

Abstract

The embodiment of the invention provides an image processing method and system based on intelligent spraying, and the method comprises the steps: constructing a first spraying image analysis model which comprises at least one coating processing network, and generating a plurality of second spraying image analysis models according to different integration sequences of a coating feature extraction unit and a spraying effect analysis unit; and performing joint model parameter learning on the plurality of second spray image analysis models according to the first sample spray image sequence, thereby not only optimizing the parameters of each model, but also further generating the plurality of second spray image analysis models after joint model parameter learning. According to the method, model parameter learning data expansion is carried out on the first sample spraying image sequence by utilizing the optimized models, and a richer and more diversified second sample spraying image sequence is generated. And finally, performing model parameter learning on the first spray image analysis model according to the second sample spray image sequence, and generating a target spray image analysis model with higher accuracy and generalization ability.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology. Specifically, it relates to an image processing method and system based on intelligent spraying. Background Art

[0002] In the field of intelligent spraying, image processing technology plays a crucial role. Traditional spraying image processing methods often rely on manual experience or simple algorithms, making it difficult to accurately and efficiently analyze the coating characteristics and spraying effects in spraying images. With the rapid development of computer vision and deep learning technologies, image processing methods based on neural networks have gradually become the mainstream, providing new ideas for spraying image processing.

[0003] However, the existing spraying image processing methods based on neural networks still have some limitations. On the one hand, a single neural network model often has difficulty fully capturing the complex features in spraying images, resulting in limited accuracy of the analysis results. On the other hand, traditional model training methods usually rely on a large amount of labeled data, and in the field of spraying image processing, it is often difficult and costly to obtain high-quality labeled data. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide an image processing method and system based on intelligent spraying.

[0005] Combined with the first aspect of this application, an image processing method based on intelligent spraying is provided, which is applied to an image processing system based on intelligent spraying. The method includes:

[0006] Obtain a first spraying image analysis model, where the first spraying image analysis model includes at least one coating processing network, and the coating processing network includes a coating feature extraction unit and a spraying effect analysis unit;

[0007] Generate a plurality of second spraying image analysis models based on different integration orders of the coating feature extraction unit and the spraying effect analysis unit in each coating processing network;

[0008] Perform joint model parameter learning on the plurality of second spraying image analysis models according to a first sample spraying image sequence to generate a plurality of second spraying image analysis models after joint model parameter learning;

[0009] Perform model parameter learning data expansion on the first sample spraying image sequence according to the plurality of second spraying image analysis models after joint model parameter learning to generate a second sample spraying image sequence;

[0010] Perform model parameter learning on the first spraying image analysis model according to the second sample spraying image sequence to generate a target spraying image analysis model.

[0011] In a possible implementation of the first aspect, generating a plurality of second spray image analysis models based on different integration orders of the coating feature extraction unit and the spraying effect analysis unit in each of the coating processing networks includes:

[0012] Obtain the number of coating processing networks in the first spray image analysis model;

[0013] Determine the model architectures of a plurality of spray image analysis models according to the number of coating processing networks, and the model architecture of each spray image analysis model corresponds to an enabled order integration of the coating feature extraction unit and the spraying effect analysis unit;

[0014] Generate a plurality of second spray image analysis models based on the model architectures of the plurality of spray image analysis models.

[0015] In a possible implementation of the first aspect, the joint model parameter learning of the plurality of second spray image analysis models based on the first sample spray image sequence to generate the plurality of second spray image analysis models after joint model parameter learning includes:

[0016] Obtain a first sample spray image sequence, where the first sample spray image sequence includes a plurality of sample spray images and spray feature annotation data corresponding to each sample spray image;

[0017] Load any one target sample spray image in the plurality of sample spray images into the plurality of second spray image analysis models respectively to generate a plurality of sample prediction results corresponding to the target sample spray image;

[0018] Optimize the neuron weight information of the plurality of second spray image analysis models based on the error between the plurality of sample prediction results and the spray feature annotation data corresponding to the target sample spray image, and the plurality of second spray image analysis models share neuron weight information;

[0019] When the plurality of second spray image analysis models do not meet the training termination condition, return to execute the step of loading any one target sample spray image into the plurality of second spray image analysis models respectively, and optimizing the neuron weight information of the plurality of second spray image analysis models based on the error between the generated plurality of sample prediction results and the corresponding spray feature annotation data;

[0020] When the plurality of second spray image analysis models meet the training termination condition, output the plurality of second spray image analysis models after joint model parameter learning.

[0021] In a possible implementation of the first aspect, optimizing the neuron weight information of the multiple second spray image analysis models based on the error between the multiple sample prediction results and the spray feature annotation data corresponding to the target sample spray image, where the multiple second spray image analysis models share the neuron weight information, includes:

[0022] Calculate the feature distances between each sample prediction result in the multiple sample prediction results and the spray feature annotation data corresponding to the target sample spray image respectively, and generate multiple training error results;

[0023] Calculate the derivative changes corresponding to each training error result, and generate multiple derivative changes;

[0024] Optimize the neuron weight information of the multiple second spray image analysis models based on the multiple derivative changes, where the multiple second spray image analysis models share the neuron weight information.

[0025] In a possible implementation of the first aspect, expanding the model parameter learning data of the first sample spray image sequence according to the multiple second spray image analysis models after learning the joint model parameters to generate a second sample spray image sequence, includes:

[0026] Expand the first sample spray image sequence according to the multiple second spray image analysis models after learning the joint model parameters to generate an expanded sample spray image sequence;

[0027] Determine multiple target expanded sample spray images from the expanded sample spray image sequence, and add the multiple target expanded sample spray images to the first sample spray image sequence to generate a second sample spray image sequence.

[0028] In a possible implementation of the first aspect, expanding the first sample spray image sequence according to the multiple second spray image analysis models after learning the joint model parameters to generate an expanded sample spray image sequence, includes:

[0029] Process each sample spray image in the first sample spray image sequence according to each second spray image analysis model after learning the joint model parameters, and generate the confidence distribution output by each second spray image analysis model after learning the joint model parameters;

[0030] Determine the expanded sample spray image sequence based on the confidence distribution output by each second spray image analysis model after learning the joint model parameters.

[0031] In a possible implementation of the first aspect, determining the extended example spray image sequence based on the confidence distribution output by the second spray image analysis model after learning each joint model parameter includes:

[0032] Determine the maximum confidence in each confidence distribution, and generate the maximum confidence output by the second spray image analysis model after learning each joint model parameter;

[0033] Determine the extended example spray image corresponding to each maximum confidence, and generate multiple extended example spray images;

[0034] Determine the extended example spray image sequence based on the multiple extended example spray images.

[0035] In a possible implementation of the first aspect, the method includes:

[0036] Obtain the target spray image data, and input the target spray image data into the target spray image analysis model to generate the corresponding target spray image analysis result.

[0037] Combined with the second aspect of the present application, there is provided an image processing system based on intelligent spraying. The image processing system based on intelligent spraying includes a machine-readable storage medium and a processor. The machine-readable storage medium stores machine-executable instructions. When the processor executes the machine-executable instructions, the image processing system based on intelligent spraying implements the foregoing image processing method based on intelligent spraying.

[0038] Combined with the third aspect of the present application, there is provided a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed, the foregoing image processing method based on intelligent spraying is implemented.

[0039] In combination with any of the above aspects, the embodiments of the present application effectively increase model diversity by constructing a first spray image analysis model including at least one coating treatment network and generating multiple second spray image analysis models according to different integration orders of the coating feature extraction unit and the spray effect analysis unit. By performing joint model parameter learning on multiple second spray image analysis models based on the first sample spray image sequence, not only are the parameters of each model optimized, but also multiple second spray image analysis models after joint model parameter learning are further generated. This method also uses these optimized models to perform model parameter learning data expansion on the first sample spray image sequence, generating a richer and more diverse second sample spray image sequence. Finally, based on the second sample spray image sequence, model parameter learning is performed on the first spray image analysis model, generating a target spray image analysis model with higher accuracy and generalization ability. This method significantly improves the efficiency and accuracy of spray image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained in combination with these drawings without creative efforts.

[0041] Figure 1 Flow diagram of the image processing method based on intelligent spraying provided by the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0043] The terms "first", "second", etc. in the specification and claims of the present invention and the above drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product, or terminal including a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or terminals.

[0044] References to "embodiments" in this specification mean that a particular feature, structure, or characteristic described in connection with the embodiments can be included in at least one embodiment of the invention. The phrase does not necessarily refer to the same embodiment each time it appears in the specification, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0045] Figure 1 The flowchart of the image processing method based on intelligent spraying provided by the embodiments of the present application is shown. It should be understood that in other embodiments, the order of some steps of the image processing method based on intelligent spraying in this embodiment can be shared with each other according to actual needs, or some of the steps can be omitted or maintained. The details of the image processing method based on intelligent spraying include:

[0046] Step S110: Obtain a first spraying image analysis model, where the first spraying image analysis model includes at least one coating processing network, and the coating processing network includes a coating feature extraction unit and a spraying effect analysis unit.

[0047] In this embodiment, in the painting process scenario of an automobile production workshop, in order to accurately analyze the painting quality of the automobile body, a first spraying image analysis model is constructed. The painting of the automobile body involves multiple coatings, such as a primer layer, a topcoat layer, etc. The quality of each coating has an important impact on the overall painting effect. The coating processing network in the first spraying image analysis model is set up to analyze these different coatings. The coating feature extraction unit is responsible for extracting various features related to the coating from the captured painting image of the automobile body. For example, for the primer layer, it may extract features such as the color uniformity and thickness distribution of the primer; for the topcoat layer, it will extract features such as the gloss and texture of the topcoat. The spraying effect analysis unit evaluates the spraying effect based on the features extracted by the coating feature extraction unit. For example, if the color uniformity of the primer is poor or the thickness is uneven, the spraying effect analysis unit can detect this situation and judge the possible impact on the overall spraying effect, such as whether it will cause poor adhesion of the topcoat or an uneven appearance. Through such a first spraying image analysis model, the painting quality of the automobile body can be initially analyzed to a certain extent, but it may have some problems such as insufficient accuracy or limited adaptation range, which need to be further optimized in subsequent steps.

[0048] Step S120: Generate a plurality of second spraying image analysis models based on different integration orders of the coating feature extraction unit and the spraying effect analysis unit in each coating processing network.

[0049] Taking the example of paint analysis in an automobile production plant, let's assume that the first paint image analysis model has three coating processing networks. There are several possible integration orders for the coating feature extraction unit and the paint effect analysis unit. The first option is to perform coating feature extraction first, followed by paint effect analysis. This is a relatively straightforward approach. In this integration order, the coating feature extraction unit first comprehensively extracts the features of each coating layer in the car exterior paint image, accurately obtaining information such as the thickness and color of each layer. This feature data is then passed to the paint effect analysis unit. The paint effect analysis unit analyzes this complete feature data and can more accurately determine the quality of the paint finish, such as the presence of paint runoff or orange peel. The second option is to perform a partial paint effect analysis first, followed by coating feature extraction. For example, after initially determining whether the painted surface has obvious defects, detailed coating feature extraction can be performed. The advantage of this approach is that if the initial analysis reveals large defects on the painted surface, a more detailed coating feature extraction may not be necessary, and the paint finish can be determined as poor, thus saving computing resources. The third scenario alternates between coating feature extraction and spray effect analysis. For example, partial features of the primer layer are extracted first, followed by a preliminary spray effect analysis. Features of the topcoat layer are then further extracted, followed by a more in-depth spray effect analysis. This approach allows for more flexible analysis of spray paint images, tailored to the characteristics of different coatings and the actual conditions of the painting process. These three different integration sequences generate three different second-pass spray image analysis models, each with its own unique analysis logic and advantages.

[0050] Step S130 , performing joint model parameter learning on the plurality of second spraying image analysis models according to the first sample spraying image sequence, and generating a plurality of second spraying image analysis models after the joint model parameter learning.

[0051] In the scenario of automotive paint quality inspection, the first sample spray image sequence is a representative image set selected from a large number of existing automotive body paint images. This image set contains automotive body images with various paint qualities, and each image has corresponding spray feature annotation data. For example, the annotation data corresponding to an image indicates that the primer layer thickness of the automotive body is uneven and there is paint running on the topcoat. This first sample spray image sequence is used for the joint model parameter learning of multiple second spray image analysis models. Taking one of the second spray image analysis models as an example, when any target sample spray image in the first sample spray image sequence is loaded into this model, the model will analyze the image according to its own structure and parameters and generate a sample prediction result. For example, the model predicts that the paint effect of the automotive body is good, but in fact the annotation data shows problems. At this time, the error between this sample prediction result and the spray feature annotation data corresponding to the target sample spray image will be calculated. For multiple second spray image analysis models, they share neuron weight information. Calculate the feature distance between each sample prediction result and the annotation data, such as by calculating the differences in color features, texture features, etc., to generate multiple training error results. Then calculate the derivative changes corresponding to each training error result. Optimize the neuron weight information of multiple second spray image analysis models based on these derivative changes. If after one optimization, multiple second spray image analysis models still do not meet the training termination conditions, such as the error between the prediction result and the annotation data is still large, then the next target sample spray image will continue to be loaded into the model, and the above error calculation, derivative calculation, and weight optimization processes will be repeated. Until multiple second spray image analysis models meet the training termination conditions, such as the error between the prediction result and the annotation data reaches a preset threshold, at this time, output multiple second spray image analysis models after joint model parameter learning.

[0052] Step S140, expand the model parameter learning data of the first sample spray image sequence according to the multiple second spray image analysis models after the joint model parameter learning to generate a second sample spray image sequence.

[0053] In the scenario of automotive paint spraying detection, multiple second spray image analysis models after joint model parameter learning already have a certain degree of accuracy. Now, based on these models, data expansion for model parameter learning of the first sample spray image sequence is to be carried out. First, each sample spray image in the first sample spray image sequence is processed according to each second spray image analysis model after joint model parameter learning. For example, for a car body paint spraying image, each model will output a confidence distribution based on its learning results. This confidence distribution reflects the model's judgment confidence level for different paint spraying features in the image. For example, for the feature of whether the primer layer thickness is uniform, the model may give a relatively high confidence, indicating that it is more certain about this judgment; while for whether the gloss of the topcoat meets the standard, it may give a relatively low confidence. Determine the expanded sample spray image sequence based on the confidence distributions output by each second spray image analysis model after joint model parameter learning. Specifically, first determine the maximum confidence in each confidence distribution. Suppose in the confidence distribution of a model, the confidence in the uniformity of the primer layer color is the highest. Then determine the expanded sample spray image corresponding to this maximum confidence. For example, according to the judgment result corresponding to this maximum confidence, find some paint spraying image features or regions that are relevant to this judgment and may not have been fully considered before. Determine these relevant image features or regions as the expanded sample spray images. By performing such operations on each model, multiple expanded sample spray images are generated. Finally, select some representative images from these expanded sample spray images and add them to the first sample spray image sequence to generate the second sample spray image sequence. The purpose of doing this is to enrich the sample data and enable subsequent model learning to be more comprehensive and accurate.

[0054] Step S150: Perform model parameter learning on the first spray image analysis model according to the second sample spray image sequence to generate a target spray image analysis model.

[0055] In the continuous improvement process of automotive paint quality analysis, the first spray image analysis model is subjected to model parameter learning using the newly generated second sample spray image sequence. The second sample spray image sequence contains more diverse and comprehensive automotive body paint images and corresponding feature annotation data. The first spray image analysis model will readjust its parameters based on this new data. For example, for new features or features that have not been fully emphasized in the automotive body paint images, the model will adjust the parameters of the coating feature extraction unit to more accurately extract these features. At the same time, for the spray effect analysis unit, the judgment criteria and logic will also be adjusted according to the new data. For example, if there are some special spray effect situations in the second sample spray image sequence, such as special texture changes caused by painting in a high-temperature environment, the model will learn this special situation and adjust the parameters to accurately identify it in subsequent analyses. By continuously learning and adjusting according to the data in the second sample spray image sequence, the accuracy and adaptability of the first spray image analysis model are continuously improved, and finally, the target spray image analysis model is generated. This target spray image analysis model can more accurately analyze the paint quality of the automotive body, and can accurately judge both conventional paint problems and spray effect in some special situations. For example, when facing automotive body paint images of different colors and different vehicle models, the target spray image analysis model can accurately extract coating features and analyze the spray effect, thus providing a more reliable basis for paint quality control in the automotive production workshop.

[0056] Based on the above steps, the embodiment of the present application effectively increases model diversity by constructing the first spray image analysis model including at least one coating processing network and generating multiple second spray image analysis models according to different integration orders of the coating feature extraction unit and the spray effect analysis unit in each coating processing network. By performing joint model parameter learning on multiple second spray image analysis models based on the first sample spray image sequence, not only are the parameters of each model optimized, but also multiple second spray image analysis models after joint model parameter learning are further generated. This method also uses these optimized models to perform model parameter learning data expansion on the first sample spray image sequence, generating a more abundant and diverse second sample spray image sequence. Finally, model parameter learning is performed on the first spray image analysis model based on the second sample spray image sequence, generating a target spray image analysis model with higher accuracy and generalization ability. This method significantly improves the efficiency and accuracy of spray image processing.

[0057] In a possible implementation manner, the generating multiple second spray image analysis models according to different integration orders of the coating feature extraction unit and the spray effect analysis unit in each coating processing network includes:

[0058] Obtain the number of coating processing networks in the first spray image analysis model.

[0059] Determine the model architectures of multiple spray image analysis models according to the number of coating processing networks, and the model architecture of each spray image analysis model corresponds to an integrated activation order of the coating feature extraction unit and the spray effect analysis unit.

[0060] Generate multiple second spray image analysis models based on the model architectures of the multiple spray image analysis models.

[0061] In a possible implementation manner, the jointly learning the model parameters of the multiple second spray image analysis models based on the first sample spray image sequence to generate multiple second spray image analysis models after jointly learning the model parameters includes:

[0062] Obtain a first sample spray image sequence, where the first sample spray image sequence includes multiple sample spray images and spray feature annotation data corresponding to each sample spray image.

[0063] Load any one target sample spray image in the multiple sample spray images into the multiple second spray image analysis models respectively to generate multiple sample prediction results corresponding to the target sample spray image.

[0064] Optimize the neuron weight information of the multiple second spray image analysis models based on the error between the multiple sample prediction results and the spray feature annotation data corresponding to the target sample spray image, and the multiple second spray image analysis models share the neuron weight information.

[0065] When the multiple second spray image analysis models do not meet the training termination condition, return to execute the step of loading any one target sample spray image into the multiple second spray image analysis models respectively and optimizing the neuron weight information of the multiple second spray image analysis models based on the error between the generated multiple sample prediction results and the corresponding spray feature annotation data.

[0066] When the multiple second spray image analysis models meet the training termination condition, output the multiple second spray image analysis models after jointly learning the model parameters.

[0067] In a possible implementation manner, the optimizing the neuron weight information of the multiple second spray image analysis models based on the error between the multiple sample prediction results and the spray feature annotation data corresponding to the target sample spray image, and the multiple second spray image analysis models share the neuron weight information includes:

[0068] Calculate the feature distances between each of the multiple sample prediction results and the spraying feature annotation data corresponding to the target sample spraying image, and generate multiple training error results.

[0069] Calculate the derivative changes corresponding to each training error result, and generate multiple derivative changes.

[0070] Optimize the neuron weight information of the multiple second spraying image analysis models based on the multiple derivative changes, and the multiple second spraying image analysis models share the neuron weight information.

[0071] In a possible implementation manner, the generating the second sample spraying image sequence by performing model parameter learning data expansion on the first sample spraying image sequence according to the multiple second spraying image analysis models after learning the joint model parameters includes:

[0072] Perform expansion on the first sample spraying image sequence according to the multiple second spraying image analysis models after learning the joint model parameters, and generate an expanded sample spraying image sequence.

[0073] Determine multiple target expanded sample spraying images from the expanded sample spraying image sequence, and add the multiple target expanded sample spraying images to the first sample spraying image sequence to generate a second sample spraying image sequence.

[0074] In a possible implementation manner, the performing expansion on the first sample spraying image sequence according to the multiple second spraying image analysis models after learning the joint model parameters to generate an expanded sample spraying image sequence includes:

[0075] Process each sample spraying image in the first sample spraying image sequence according to each second spraying image analysis model after learning the joint model parameters, and generate a confidence distribution output by each second spraying image analysis model after learning the joint model parameters.

[0076] Determine the expanded sample spraying image sequence based on the confidence distribution output by each second spraying image analysis model after learning the joint model parameters.

[0077] In a possible implementation manner, the determining the expanded sample spraying image sequence based on the confidence distribution output by each second spraying image analysis model after learning the joint model parameters includes:

[0078] Determine the maximum confidence in each confidence distribution, and generate the maximum confidence output by each second spraying image analysis model after learning the joint model parameters.

[0079] Determine the extended example spray painting images corresponding to each maximum confidence, and generate multiple extended example spray painting images.

[0080] Determine an extended example spray painting image sequence based on the multiple extended example spray painting images.

[0081] In a possible implementation manner, the method includes:

[0082] Obtain target spray painting image data, and input the target spray painting image data into the target spray painting image analysis model to generate a corresponding target spray painting image analysis result.

[0083] In this embodiment, multiple second spray painting image analysis models are generated based on different integration orders of the coating feature extraction unit and the spray painting effect analysis unit in each coating processing network:

[0084] In automobile production, the quality inspection of the paint spraying on the automobile body is crucial. The first spray painting image analysis model is constructed for analyzing spray painting images and contains several coating processing networks. First, obtain the number of coating processing networks in the first spray painting image analysis model. Assume there are three coating processing networks in this model for automobile paint spraying inspection. These three coating processing networks are respectively for the primer layer, the intermediate paint layer, and the topcoat layer in automobile paint spraying. Determine the model architectures of multiple spray painting image analysis models according to the number of these three coating processing networks. For the coating feature extraction unit and the spray painting effect analysis unit in each coating processing network, different enabled order integrations will correspond to different model architectures. For example, in a model architecture for the coating processing network of the primer layer, first operate the coating feature extraction unit to extract features such as primer color, primer thickness, and primer surface flatness from the primer layer spray painting image, and then transfer these extracted features to the spray painting effect analysis unit, which determines whether the spray painting effect of the primer layer is qualified according to these features, whether there are problems such as uneven color, too thin or too thick thickness, and uneven surface. For the coating processing network of the intermediate paint layer, first conduct a preliminary judgment by the spray painting effect analysis unit, such as first judging whether there are obvious defects or problems with the bonding to the primer layer, and then operate the coating feature extraction unit to extract in detail the color features of the intermediate paint layer, the transition features with the primer layer and the topcoat layer, etc. For the coating processing network of the topcoat layer, adopt the method of alternately operating the coating feature extraction unit and the spray painting effect analysis unit. First, extract some features of the topcoat layer such as glossiness, conduct a preliminary spray painting effect analysis, then further extract other features such as the texture features of the topcoat layer, and conduct a more in-depth spray painting effect analysis again. Based on the model architectures determined by these three different enabled order integrations for each coating processing network, generate multiple second spray painting image analysis models, and each model can analyze the automobile body spray painting image from different logics and perspectives.

[0085] Perform joint model parameter learning on the multiple second spray painting image analysis models according to the first sample spray painting image sequence to generate multiple second spray painting image analysis models after joint model parameter learning:

[0086] In the scenario of automobile spray painting quality inspection, the first sample spray painting image sequence is a representative image set carefully selected from a large number of existing automobile body spray painting images. Each sample spray painting image in this sequence has corresponding spray painting feature annotation data. For example, a sample spray painting image is a partial spray painting image of a certain automobile body, and its corresponding spray painting feature annotation data details that the primer layer thickness is a specific value, the color uniformity is within a certain range, there is a slight running paint phenomenon on the topcoat layer, the glossiness is at a certain level, etc. Load any one of these sample spray painting images, i.e., the target sample spray painting image, into the multiple second spray painting image analysis models respectively. For example, select a spray painting image of a car body door part as the target sample spray painting image and load it into the multiple second spray painting image analysis models generated previously. Each second spray painting image analysis model analyzes this target sample spray painting image according to its own structure and parameters, thereby generating multiple sample prediction results corresponding to the target sample spray painting image. One model may predict that the primer layer thickness is uniform but the color uniformity is slightly poor, the running paint phenomenon on the topcoat layer is not obvious and the glossiness is good; another model may have different prediction results, such as there are small fluctuations in the primer layer thickness, the color uniformity is qualified, there is a slight running paint on the topcoat layer and the glossiness is slightly lower, etc. Then, optimize the neuron weight information of the multiple second spray painting image analysis models based on the errors between these multiple sample prediction results and the spray painting feature annotation data corresponding to the target sample spray painting image, and these second spray painting image analysis models share the neuron weight information. When the multiple second spray painting image analysis models do not meet the training termination conditions, for example, the errors between the prediction results and the annotation data are still relatively large, such as the difference between the predicted value and the annotated value of the primer layer thickness exceeds a certain range, or the judgment of the running paint phenomenon on the topcoat layer does not match the annotation, etc., then return to execute the step of loading any one of the target sample spray painting images into the multiple second spray painting image analysis models respectively and optimizing the neuron weight information of the multiple second spray painting image analysis models based on the errors between the generated multiple sample prediction results and the corresponding spray painting feature annotation data. Continuously repeat this process until the multiple second spray painting image analysis models meet the training termination conditions, for example, the errors between the prediction results and the annotation data are reduced to a preset very small range on all key features. At this time, output the multiple second spray painting image analysis models after joint model parameter learning.

[0087] Optimize the neuron weight information of the multiple second spray image analysis models based on the error between the multiple sample prediction results and the spray feature annotation data corresponding to the target sample spray image, where the multiple second spray image analysis models share the neuron weight information:

[0088] In the analysis of automotive spray painting images, take the previously mentioned automotive body spray painting image as an example. For the target sample spray image, after multiple second spray image analysis models generate multiple sample prediction results, it is necessary to calculate the feature distances between each sample prediction result in the multiple sample prediction results and the spray feature annotation data corresponding to the target sample spray image respectively, and generate multiple training error results. For example, for the feature of the primer layer thickness, a sample prediction result is that the thickness is 2 mm, while the annotation data is 1.8 mm. By calculating the absolute value of the difference between the two, etc., the feature distance of this feature is obtained, and then a training error result is generated. For the feature of the topcoat layer glossiness, the sample prediction result is that the glossiness level is 3, and the annotation data is 2. Similarly, the feature distance is calculated to obtain the corresponding training error result. Then calculate the derivative changes corresponding to each training error result to generate multiple derivative changes. For example, according to a specific error calculation function and optimization algorithm, calculate the derivative change for the training error result of the primer layer thickness, and also calculate the corresponding derivative change for the training error result of the topcoat layer glossiness. Optimize the neuron weight information of the multiple second spray image analysis models based on these multiple derivative changes. Since the multiple second spray image analysis models share the neuron weight information, optimizing the weight information of one model will affect other models. For example, if it is found during the optimization process that the prediction error of the primer layer thickness is relatively large, adjust the neuron weight information according to the derivative change, so that the primer layer thickness can be predicted more accurately in subsequent analyses. At the same time, this adjustment of the weight information will also affect other models in the prediction of the primer layer thickness, thereby overall improving the analysis accuracy of the multiple second spray image analysis models for automotive body spray painting images.

[0089] Expand the model parameter learning data of the first sample spray image sequence according to the multiple second spray image analysis models after learning the joint model parameters, and generate a second sample spray image sequence:

[0090] In the scenario of automotive paint quality inspection, multiple second spray image analysis models after joint model parameter learning are used to expand the first sample spray image sequence. First, the first sample spray image sequence is expanded according to the multiple second spray image analysis models after joint model parameter learning to generate an expanded sample spray image sequence. For example, for a spray paint image of the top of a car body shell in the first sample spray image sequence, each second spray image analysis model after joint model parameter learning processes it. One model may analyze the features of the primer layer, intermediate paint layer, and topcoat layer of this image according to the parameters and algorithms it has learned, and then output a confidence distribution. This confidence distribution may indicate that the confidence in the judgment of the primer layer thickness feature is 0.8, the confidence in the judgment of the bonding situation between the intermediate paint layer and the primer layer is 0.7, the confidence in the judgment of the glossiness of the topcoat layer is 0.9, etc. The expanded sample spray image sequence is determined based on the confidence distributions output by each second spray image analysis model after joint model parameter learning. First, determine the maximum confidence in each confidence distribution. For example, the 0.9 corresponding to the glossiness of the topcoat layer in the above example is the maximum confidence output by this model. Then determine the expanded sample spray image corresponding to each maximum confidence. For example, according to this maximum confidence of the glossiness of the topcoat layer, find some image features or regions related to the glossiness of the topcoat layer that may not have been fully considered before, and determine these as the expanded sample spray images. By performing such operations on each model, multiple expanded sample spray images are generated. Finally, multiple target expanded sample spray images are determined from the expanded sample spray image sequence. For example, select those expanded sample spray images with a high degree of relevance to key spray paint features and that are helpful for improving the accuracy of the model, and add these multiple target expanded sample spray images to the first sample spray image sequence to generate the second sample spray image sequence.

[0091] Model parameter learning is performed on the first spray image analysis model according to the second sample spray image sequence to generate a target spray image analysis model:

[0092] In the scenario of automobile spray painting quality inspection, the second sample spray painting image sequence contains richer and more comprehensive automobile body spray painting images and corresponding feature annotation data. The first spray painting image analysis model is used to learn model parameters based on this second sample spray painting image sequence. For the coating feature extraction unit in the first spray painting image analysis model, the parameters are adjusted according to the new data in the second sample spray painting image sequence. For example, if there are some special automobile body spray painting images in the second sample spray painting image sequence, such as automobile body images spray painted in a specific environment, the primer layer in these images may present some new features, and the coating feature extraction unit of the first spray painting image analysis model will adjust the parameters according to these new features to more accurately extract the features of the primer layer. The same is true for the spray painting effect analysis unit, and the judgment criteria and logic are adjusted according to the data in the second sample spray painting image sequence. For example, there are some automobile body spray painting images in the second sample spray painting image sequence with special paint running phenomena caused by new spray painting processes or environmental factors, and the spray painting effect analysis unit of the first spray painting image analysis model will learn these new situations and adjust the criteria for judging whether the paint running phenomenon is qualified. By continuously learning and adjusting according to the data in the second sample spray painting image sequence, the accuracy and adaptability of the first spray painting image analysis model are continuously improved, and finally a target spray painting image analysis model is generated. This target spray painting image analysis model can more accurately analyze the spray painting quality of the automobile body, and can accurately judge both conventional spray painting problems and spray painting effects in special situations.

[0093] Obtain target spray painting image data and input the target spray painting image data into the target spray painting image analysis model to generate corresponding target spray painting image analysis results:

[0094] In the quality inspection process of an automotive production workshop, target spray painting image data of the vehicle body is obtained. This target spray painting image data is a high-resolution image captured by a dedicated image acquisition device of the painted surface of the vehicle body. The target spray painting image data is input into the previously generated target spray painting image analysis model. The target spray painting image analysis model analyzes the target spray painting image data based on the parameters and algorithms that have been learned and optimized. For example, the model first extracts various features of the primer layer, intermediate paint layer, and topcoat layer in the target spray painting image through a coating feature extraction unit, including color, thickness, texture, glossiness, etc. Then, a spray painting effect analysis unit determines whether the spray painting effect is qualified based on these extracted features. If the color uniformity of the primer layer is within the qualified range, the thickness meets the standard, the intermediate paint layer adheres well to the upper and lower layers, the glossiness of the topcoat layer is appropriate, and there are no phenomena such as running paint or orange peel, then the model will generate an analysis result of the target spray painting image indicating that the spray painting effect is qualified; if there are any features that do not meet the standards, the model will accurately point out the problems, such as uneven thickness of a certain paint layer or running paint in the topcoat layer, etc., thereby providing an accurate basis for the evaluation of the spray painting quality of the vehicle body.

[0095] In the above embodiments, the intelligent spray-based image processing system for implementing the above method embodiments has at least one processor, a control module (chipset) coupled to at least one of the (at least one) processors, a memory coupled to the control module, a non-volatile memory (NVM) / storage device coupled to the control module, at least one load / output device coupled to the control module, and a network interface coupled to the control module.

[0096] The processor may include at least one single-core or multi-core processor, and the processor may include any combination of general-purpose processors or dedicated processors (such as graphics processors, application processors, baseband processors, etc.). For some alternative embodiments, the intelligent spray-based image processing system can act as an electronic device such as a gateway described in the embodiments of the present application.

[0097] For some alternative embodiments, the intelligent spray-based image processing system may include at least one computer-readable medium (e.g., a memory or NVM / storage device) having instructions and at least one processor integrated with the at least one computer-readable medium and configured to execute the instructions to implement modules and thus perform the actions described in the present disclosure.

[0098] For one embodiment, the control module may include any suitable interface controller to provide any suitable interface to at least one of the (at least one) processors and / or any suitable device or component communicating with the control module.

[0099] The control module may include a memory controller module to provide an interface to the memory. The memory controller module can be a hardware module, a software module, and / or a firmware module.

[0100] The memory can be used, for example, to load and store data and / or instructions for an intelligent spray-based image processing system. For one embodiment, the memory may include any suitable volatile memory, e.g., a suitable DRAM.

[0101] For one embodiment, the control module may include at least one load-to / output controller to provide an interface to the NVM / storage device and the (at least one) load-to / output device.

[0102] For example, the NVM / storage device can be used to store data and / or instructions. The NVM / storage device may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (at least one) non-volatile storage device (e.g., at least one hard disk drive (HDD), at least one compact disc (CD) drive, and / or at least one digital versatile disc (DVD) drive).

[0103] The NVM / storage device may include storage resources that are physically part of the device on which the intelligent spray-based image processing system is installed, or it may be accessible by the device without being part of the device. For example, the NVM / storage device can be accessed via the (at least one) load-to / output device based on a network.

[0104] (The at least one) load-to / output device can provide an interface for the intelligent spray-based image processing system to communicate with any other suitable device. The load-to / output device may include communication components, spelling components, sensor components, etc. The network interface can provide an interface for the intelligent spray-based image processing system to communicate based on at least one network. The intelligent spray-based image processing system can wirelessly communicate with at least one component of the wireless network based on any prior and / or protocol in at least one wireless network prior and / or protocol, e.g., accessing a wireless network based on a communication prior.

[0105] For one embodiment, at least one of the (at least one) processors can be logically loaded together with at least one controller of the control module (e.g., the memory controller module). For one embodiment, at least one of the (at least one) processors can be logically loaded together with at least one controller of the control module to form a system-level load. For one embodiment, at least one of the (at least one) processors can be logically integrated with at least one controller of the control module on the same die. For one embodiment, at least one of the (at least one) processors can be logically integrated with at least one controller of the control module on the same die to form a system-on-chip (SoC).

[0106] The above has introduced the embodiments of the present application in detail. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, based on the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

[0107] An embodiment of the present invention discloses a computer-readable storage medium that stores a computer program for electronic data exchange. Among them, the computer program enables the computer to execute the steps in the image processing method based on intelligent spraying described in the foregoing embodiments.

[0108] An embodiment of the present invention discloses a computer program product. The computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable the computer to execute the steps in the image processing method based on intelligent spraying described in the foregoing embodiments.

[0109] The device embodiments described above are only illustrative. The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0110] Through the above specific description of the embodiments, those skilled in the art can clearly understand that each implementation can be realized by means of software plus a necessary general hardware platform, and of course, it can also be realized by hardware. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, and the storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disk memories, tape memories, or any other medium that can be used for a computer to have or store data.

[0111] Finally, it should be noted that: the above-disclosed is only the preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method based on intelligent spraying, characterized in that, The method includes: Obtaining a first spray image analysis model, where the first spray image analysis model includes at least one coating processing network, and the coating processing network includes a coating feature extraction unit and a spraying effect analysis unit; Generating a plurality of second spray image analysis models based on different integration orders of the coating feature extraction unit and the spraying effect analysis unit in each of the coating processing networks; Performing joint model parameter learning on the plurality of second spray image analysis models according to a first sample spray image sequence to generate a plurality of second spray image analysis models after joint model parameter learning; Performing model parameter learning data expansion on the first sample spray image sequence according to the plurality of second spray image analysis models after joint model parameter learning to generate a second sample spray image sequence; Performing model parameter learning on the first spray image analysis model according to the second sample spray image sequence to generate a target spray image analysis model.

2. The image processing method based on intelligent spraying according to claim 1, wherein, The generating a plurality of second spray image analysis models based on different integration orders of the coating feature extraction unit and the spraying effect analysis unit in each of the coating processing networks includes: Obtaining the number of coating processing networks in the first spray image analysis model; Determining the model architectures of a plurality of spray image analysis models according to the number of coating processing networks, and the model architecture of each spray image analysis model corresponds to an enabled order integration of the coating feature extraction unit and the spraying effect analysis unit; Generating a plurality of second spray image analysis models based on the model architectures of the plurality of spray image analysis models.

3. The image processing method based on intelligent spraying according to claim 1 or 2, characterized in that The performing joint model parameter learning on the plurality of second spray image analysis models according to a first sample spray image sequence to generate a plurality of second spray image analysis models after joint model parameter learning includes: Obtaining a first sample spray image sequence, where the first sample spray image sequence includes a plurality of sample spray images and spraying feature annotation data corresponding to each sample spray image; Loading any one target sample spray image in the plurality of sample spray images into the plurality of second spray image analysis models respectively to generate a plurality of sample prediction results corresponding to the target sample spray image; Optimizing the neuron weight information of the plurality of second spray image analysis models based on the error between the plurality of sample prediction results and the spraying feature annotation data corresponding to the target sample spray image, and the plurality of second spray image analysis models share neuron weight information; When the plurality of second spray image analysis models do not meet the training termination condition, returning to execute the step of loading any one target sample spray image into the plurality of second spray image analysis models respectively and optimizing the neuron weight information of the plurality of second spray image analysis models based on the error between the generated plurality of sample prediction results and the corresponding spraying feature annotation data; When the plurality of second spray image analysis models meet the training termination condition, outputting a plurality of second spray image analysis models after joint model parameter learning.

4. The image processing method based on intelligent spraying according to claim 3, characterized in that Optimizing the neuron weight information of the multiple second spray image analysis models based on the error between the multiple sample prediction results and the spray feature annotation data corresponding to the target sample spray image, where the multiple second spray image analysis models share the neuron weight information, includes: Calculating the feature distances between each sample prediction result in the multiple sample prediction results and the spray feature annotation data corresponding to the target sample spray image respectively, and generating multiple training error results; Calculating the derivative changes corresponding to each training error result, and generating multiple derivative changes; Optimizing the neuron weight information of the multiple second spray image analysis models based on the multiple derivative changes, where the multiple second spray image analysis models share the neuron weight information.

5. The image processing method based on intelligent spraying according to claim 1, wherein The generating of the second sample spray image sequence by expanding the model parameter learning data of the first sample spray image sequence according to the multiple second spray image analysis models after learning the joint model parameters includes: Expanding the first sample spray image sequence according to the multiple second spray image analysis models after learning the joint model parameters, and generating an expanded sample spray image sequence; Determining multiple target expanded sample spray images from the expanded sample spray image sequence, and adding the multiple target expanded sample spray images to the first sample spray image sequence to generate the second sample spray image sequence.

6. The image processing method based on intelligent spraying according to claim 5, characterized in that, The expanding of the first sample spray image sequence according to the multiple second spray image analysis models after learning the joint model parameters to generate an expanded sample spray image sequence includes: Processing each sample spray image in the first sample spray image sequence according to each second spray image analysis model after learning the joint model parameters, and generating the confidence distribution output by each second spray image analysis model after learning the joint model parameters; Determining the expanded sample spray image sequence based on the confidence distribution output by each second spray image analysis model after learning the joint model parameters.

7. The image processing method based on intelligent spraying according to claim 6, characterized in that The determining of the expanded sample spray image sequence based on the confidence distribution output by each second spray image analysis model after learning the joint model parameters includes: Determining the maximum confidence in each confidence distribution, and generating the maximum confidence output by each second spray image analysis model after learning the joint model parameters; Determining the expanded sample spray image corresponding to each maximum confidence, and generating multiple expanded sample spray images; Determining the expanded sample spray image sequence based on the multiple expanded sample spray images.

8. The image processing method based on intelligent spraying according to claim 1, wherein The method includes: Obtaining target spray image data, and inputting the target spray image data into the target spray image analysis model to generate a corresponding target spray image analysis result.

9. A computer-readable storage medium, characterized in that, The machine-executable instructions are stored in the computer-readable storage medium, and when the machine-executable instructions are executed by a computer, the image processing method based on intelligent spraying according to any one of claims 1-8 is implemented.

10. An image processing system based on intelligent spraying, characterized in that, It includes a processor and a computer-readable storage medium, and machine-executable instructions are stored in the computer-readable storage medium. When the machine-executable instructions are executed by a computer, the image processing method based on intelligent spraying described in any one of claims 1-8 is implemented.

Images