Machine vision deep learning model training method based on sample feature distribution diagram, electronic device, and storage medium

Abstract

The application discloses a machine vision deep learning model training method based on a sample feature distribution diagram, an electronic device and a storage medium. The machine vision deep learning model training method comprises the following steps: S1, acquiring a sample set about material surface defects and a machine vision deep learning model trained based on the sample set; S2, verifying the sample set by the machine vision deep learning model and establishing a sample feature space distribution diagram according to the verified feature information result; S3, based on the visual characteristics of the sample feature space distribution diagram, locating to the sample feature of the problem and correcting the same to obtain a corrected sample set; and S4, retraining the machine vision deep learning model based on the corrected sample set. The application has the beneficial effect of improving the inference effect of the machine vision deep learning model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine vision deep learning, in particular, a machine vision deep learning model training method based on sample feature distribution graph, electronic equipment and storage medium. BACKGROUND

[0002] In the process of landing the machine vision deep learning project related to material surface defects, sample labeling and sample cleaning work are crucial. However, it also requires a lot of time and manpower, and is prone to errors. Once an error occurs, it will affect the training and inference effect of the model, and it is difficult to troubleshoot in the later stage. The specific performance is as follows: in the early stage of the project, the user provides the original sample set, and needs to label the feature categories according to the original sample set. With the iteration of the project, the newly added sample features are continuously added to the sample set, and factors such as changes of labeling personnel, changes of labeling standards, and changes of detection standards accumulated for a long time cause the sample set to become more and more unstable. Once the trained model does not meet the expectations, it takes a lot of time and manpower to clean up the samples, and it is difficult to locate the problem. SUMMARY

[0003] The technical problem to be solved by the present application is that the existing machine vision deep learning model is unstable due to the iteration of the sample set, and the training does not meet the expectations. A machine vision deep learning model training method based on sample feature distribution graph, electronic equipment and storage medium are provided.

[0004] In order to solve the above technical problems, the technical scheme of the present application is as follows: a machine vision deep learning model training method based on sample feature distribution graph, comprising:

[0005] Step S1, obtaining a sample set related to material surface defects and a machine vision deep learning model trained based on the sample set;

[0006] Step S2, verifying the sample set by the machine vision deep learning model and establishing a sample feature space distribution graph according to the feature information result of the verification: the distribution point pattern of the sample feature in the sample feature space distribution graph is determined according to the feature categories determined by the user, and the distribution point position of the sample feature in the sample feature space distribution graph is determined by applying manifold dimension reduction technology to map the sample feature from high-dimensional space to low-dimensional space;

[0007] Step S3, based on the visual characteristics of the sample feature space distribution graph, locating the sample feature of the problem and correcting it, obtaining the corrected sample set; and,

[0008] Step S4, retraining the machine vision deep learning model based on the corrected sample set.

[0009] As a preferred solution of the machine vision deep learning model training method, the material surface defect is a film material surface defect sample, a plate material surface defect sample or a sheet material surface defect sample.

[0010] As a preferred solution of the machine vision deep learning model training method, in step S3, if part of the sample features in the sample feature space distribution diagram fall within the defined region of other feature categories, it is determined that the machine vision deep learning model is incorrect and the incorrect user is corrected.

[0011] As a preferred solution of the machine vision deep learning model training method, in step S3, if the defined region ranges of two feature categories in the sample feature space distribution diagram overlap or are similar, the feature categories of all sample features within the defined regions of the two feature categories are corrected to the same feature category.

[0012] As a preferred solution of the machine vision deep learning model training method, in step S3, if part of the sample features in the sample feature space distribution diagram is far away from the defined region of all feature categories, the feature category of the part of the sample features is corrected to a new feature category.

[0013] As a preferred solution of the machine vision deep learning model training method, in step S3, if the number of sample features of the new feature category is too small, the number is appropriately expanded.

[0014] As a preferred solution of the machine vision deep learning model training method, in step S2, the distribution point position of the sample features missed by the machine vision deep learning model in the sample feature space distribution diagram is determined according to the defined region where the sample features of the same feature category that are not missed by the machine vision deep learning model are located, and the distribution point pattern of the missed sample features and the non-missed sample features is distinguished. In step S3, if part of the sample features in the sample feature space distribution diagram are distinguished, the reason for the missed detection of the part of the sample features is checked or the marking standard of the machine vision deep learning model for the part of the sample features is adjusted.

[0015] Another technical solution of the present application is as follows: an electronic device comprising a processor and a memory. The memory stores at least one instruction. The instruction is loaded and executed by the processor to implement the operation performed by the machine vision deep learning model training method based on the sample feature distribution diagram.

[0016] Still another technical solution of the present application is as follows: a storage medium, wherein at least one instruction is stored. The instruction is loaded and executed by a processor to implement the operation performed by the machine vision deep learning model training method based on a sample feature distribution map.

[0017] Compared with the prior art, the present application has at least the following beneficial effects: based on the visual characteristics of the sample feature space distribution map, the current state of the sample set can be intuitively understood, the sample features of the problem can be quickly located, reliable basis is provided for adjusting the sample set, and a large amount of time and labor cost can be saved. At the same time, the machine vision deep learning model trained based on the corrected sample set has better reasoning effect.

[0018] In addition to the technical problems solved by the present application, the technical features constituting the technical solutions, and the beneficial effects brought by these technical features described above, other technical problems solved by the present application, other technical features included in the technical solutions, and the beneficial effects brought by these technical features will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a method flowchart of an embodiment of the machine vision deep learning model training method of the present application.

[0020] Figure 2 is a schematic diagram of a sample set (determined by a user to determine a feature category) of an embodiment of the machine vision deep learning model training method of the present application.

[0021] Figure 3 is a schematic diagram of a sample feature space distribution map of an embodiment of the machine vision deep learning model training method of the present application.

[0022] Figure 4 is a schematic diagram of a sample feature space distribution map (corrected ideal sample set) of an embodiment of the machine vision deep learning model training method of the present application.

[0023] Figure 5 is a structural schematic diagram of an electronic device of an embodiment of the defect contour generation method of the present application. DETAILED DESCRIPTION

[0024] The present application will be further described in detail below by specific embodiments in conjunction with the accompanying drawings. It should be noted that the description of these embodiments is used to help understand the present application, but does not constitute a limitation on the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0025] Reference is made to Figure 1This application provides a method for training a machine vision deep learning model based on a sample feature distribution map. The machine vision deep learning model training method includes:

[0026] Step S1: Obtain a sample set of material surface defects and a machine vision deep learning model trained based on the sample set. The material surface defects are samples of membrane surface defects, plate surface defects, or sheet surface defects. The sample set includes sample features, the feature categories of which are determined by the user (i.e., the user initially labels the sample features). The sample features include: original sample features from the project initiation phase and newly added sample features from the project iteration phase. In other embodiments, the sample features may only include the original sample features from the project initiation phase, excluding newly added sample features from the project iteration phase.

[0027] Step S2 involves using the machine vision deep learning model to examine the sample set and establishing a sample feature space distribution map based on the examined feature information. The distribution pattern of the sample features in the sample feature space distribution map is determined according to the feature categories defined by the user. Sample features of the same feature category have the same distribution pattern, while sample features of different feature categories have different distribution pattern. The location of the sample features in the sample feature space distribution map is determined by applying manifold dimensionality reduction techniques (including but not limited to t-SNE, UMAP, etc.) to map the sample features from a high-dimensional space to a low-dimensional space.

[0028] The machine vision deep learning model may miss some sample features. Since the missed sample features do not have any detected feature information, the distribution points of the missed sample features in the sample feature space distribution map are determined based on the defined regions of the same feature categories that the machine vision deep learning model has not missed. The distribution point patterns of the missed sample features and the not missed sample features are differentiated to make it easier for users to identify which sample features are missed and which are not missed.

[0029] See Figure 2 and Figure 3, the feature categories of the sample features in the sample set are determined by the user as four categories A, B, C and D respectively. In the sample feature space distribution map, different gray levels of solid line dots represent the distribution point patterns of the sample features of each feature category. Specifically, the gray level of category A is the lightest, the gray level of category B is the second lightest, the gray level of category C is the second darkest, and the gray level of category D is the darkest. The missed sample features of category A are placed in the defined region of the non-missed sample features of category A, and the distribution point pattern of the missed sample features of category A is a dotted line dot, which is different from the solid line dot of the non-missed sample features of category A.

[0030] Step S3, based on the visualization characteristics of the sample feature space distribution map, the user or the computer can quickly locate the problem sample features and correct them to obtain the corrected sample set.

[0031] In the sample feature space distribution map, if the distribution point patterns of part of the sample features are differentiated (i.e., there is a missed detection situation of the machine vision deep learning model), the missed detection reason of the part of the sample features is checked or the determination standard of the machine vision deep learning model for the part of the sample features is adjusted.

[0032] In the sample feature space distribution map, if part of the sample features falls in the defined region of other feature categories (i.e., the feature category of the part of the sample features marked by the user is inconsistent with the feature category verified by the machine vision deep learning model), the error of the machine vision deep learning model and the user is determined and the error is corrected. If the error is the user, the feature category of the part of the sample features is corrected to the feature category determined by the machine vision deep learning model. If the error is the machine vision deep learning model, the determination standard of the machine vision deep learning model for the part of the sample features is adjusted.

[0033] In the sample feature space distribution map, if the defined regions of two feature categories overlap or are similar, the feature categories of all sample features in the defined regions of the two feature categories are corrected to the same feature category.

[0034] In the sample feature space distribution map, if part of the sample features is far away from the defined regions of all feature categories, the feature category of the part of the sample features is corrected to a new feature category. Further, if the number of the part of the sample features is small, the sample features of the new feature category are expanded, so as to ensure the balance of the sample features of each feature category.

[0035] Referring to Figure 3, I region shows the missed sample features (user label is A class). The distribution point pattern of the missed sample features is dotted circle point, and the distribution point pattern of the un-missed sample features is solid circle point. At this time, the reason for missing the sample features is checked or the marking standard of the machine vision deep learning model about the part of sample features is adjusted.

[0036] Referring to Figure 3 , II region shows that the sample features of B class fall in the defined region of D class, and the feature class of the sample features is corrected to D class.

[0037] Referring to Figure 3 , III region shows that the defined region of B class and the defined region of C class coincide, and the feature classes of the sample features inside the defined region of B class and the defined region of C class are unified.

[0038] Referring to Figure 3 , IV region and V region show that the distribution points of part of the sample features are far away from the A, B, C and D class regions, and the feature classes of the sample features are corrected to the newly added feature classes.

[0039] Referring to Figure 3 , IV region shows that the number of sample features is too small, and the number of sample features of the feature class is appropriately increased.

[0040] Referring to Figure 4 , the figure shows the corrected sample set, and A, B, C, D and E classes are re-determined.

[0041] Step S4, based on the corrected sample set, the machine vision deep learning model is re-trained. It is verified that the inference effect of the re-trained machine vision deep learning model is significantly improved compared with before.

[0042] Referring to Figure 5 , the embodiment of the application also provides an electronic device. The electronic device comprises a processor and a memory. The processor is connected with the memory. The processor is used to execute the computer program stored in the memory, so that the electronic device executes the machine vision deep learning model training method based on the sample feature distribution diagram.

[0043] In particular implementation, the processor can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; or a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0044] In particular implementation, the memory includes a ROM, a RAM, a disk, a U disk, a memory card or an optical disk, etc. various media capable of storing program codes.

[0045] Optionally, the electronic device further includes a display. The display is respectively connected with the memory and the processor in communication. The display is configured to display a related graphical user interface (GUI) of the machine vision deep learning model training method based on the sample feature distribution diagram.

[0046] The embodiment of the present application further provides a storage medium. The storage medium stores a computer program. The computer program is executed by a processor to implement the machine vision deep learning model training method based on the sample feature distribution diagram.

[0047] The above only expresses the embodiments of the present application, which are described in detail, but cannot be understood as the limitation of the patent scope of the present application. It should be noted that, for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for training a machine vision deep learning model based on a sample feature distribution map, characterized in that, The method comprises the following steps: Step S1, obtaining a sample set of material surface defects and a machine vision deep learning model trained based on the sample set, wherein the material surface defects are film surface defect samples, plate surface defect samples or sheet surface defect samples, and the sample set comprises sample features, and the feature categories of the sample features are determined by a user, wherein the sample features comprise original sample features in a project initiation stage and newly added sample features in a project iteration stage; Step S2, checking the sample set by the machine vision deep learning model and establishing a sample feature space distribution map according to the checked feature information results: the distribution point pattern of the sample features in the sample feature space distribution map is determined according to the feature categories determined by the user, and the distribution point positions of the sample features in the sample feature space distribution map are determined by applying a manifold dimension reduction technology to map the sample features from a high-dimensional space to a low-dimensional space; Step S3, based on the visual characteristics of the sample feature space distribution map, locating the sample features with problems and correcting them to obtain a corrected sample set; and Step S4, retraining the machine vision deep learning model based on the corrected sample set; In step S3, if some sample features in the sample feature space distribution map fall within the defined regions of other feature categories, it is determined that the machine vision deep learning model and the user are wrong, and the wrong one is corrected: if the wrong one is the user, the feature categories of the sample features are corrected to the feature categories determined by the machine vision deep learning model; if the wrong one is the machine vision deep learning model, the determination criteria of the machine vision deep learning model for the sample features are adjusted; In step S3, if the defined regions of two feature categories overlap or are similar in the sample feature space distribution map, the feature categories of all sample features within the defined regions of the two feature categories are corrected to the same feature category; In step S3, if some sample features in the sample feature space distribution map are far away from the defined regions of all feature categories, the feature categories of the sample features are corrected to a newly added feature category; if the number of sample features of the newly added feature category is too small, the number is appropriately expanded; In step S2, the distribution point positions of the sample features missed by the machine vision deep learning model in the sample feature space distribution map are determined according to the defined regions of the same feature categories of the sample features not missed by the machine vision deep learning model, and the distribution point patterns of the missed sample features and the sample features not missed are differentiated; In step S3, if the distribution point patterns of some sample features in the sample feature space distribution map are differentiated, the reasons for the missed sample features are checked or the labeling criteria of the machine vision deep learning model for the sample features are adjusted. 2.An electronic device comprising a processor and a memory in which at least one instruction is stored, wherein the at least one instruction, when executed by the processor, causes the electronic device to: The instructions are loaded and executed by the processor to implement the operations performed by the machine vision deep learning model training method based on a sample feature distribution map according to claim 1.

3. A storage medium having stored therein at least one instruction, characterized in that, The instructions are loaded and executed by the processor to implement the operations performed by the machine vision deep learning model training method based on a sample feature distribution map according to claim 1.

Images