Method and device for detecting complex graph through laser scanning

Abstract

The invention provides a method for detecting a complex graph through laser scanning, and the method comprises the steps: obtaining training data, training a neural network according to the training data, and enabling the neural network to have the capability of recognizing color information and shape information; wherein the color information and the shape information are identified through different streams; obtaining a target image, and obtaining target color information and target shape information of the target image according to the trained neural network; and performing feature fusion on the target color information and the target shape information to obtain feature information of the target image. Through the laser scanning detection method of the double-flow model, the problems of low detection speed and insufficient precision in the prior art are solved. The color and the shape of the scanned object are respectively analyzed through the double-flow model, so that the detection speed and the detection precision are remarkably improved.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of image recognition, and particularly relates to a method and device for laser scanning to detect complex graphics. Background Art

[0002] Currently, in the process of industrial part manufacturing, terminal detection usually relies on manual or simple automated detection equipment. These methods have problems such as slow detection speed, insufficient detection accuracy, and easy omission of small defects. Especially when facing complex graphics and diverse parts, traditional data models often cannot complete the detection task quickly and accurately, resulting in low production efficiency and increased costs. Summary of the Invention

[0003] In view of the above problems, the present application is proposed to provide a method and device for laser scanning to detect complex graphics that can overcome or at least partially solve the above problems, including:

[0004] A method for laser scanning to detect complex graphics, including:

[0005] Obtain training data, and train a neural network based on the training data so that the neural network has the ability to recognize color information and shape information; wherein, the color information and the shape information are recognized through different streams;

[0006] Obtain a target image, and obtain the target color information and target shape information of the target image based on the trained neural network;

[0007] Fuse the target color information and the target shape information to obtain the characteristic information of the target image.

[0008] Further, the step of obtaining training data, training a neural network based on the training data so that the neural network has the ability to recognize color information and shape information; wherein, the color information and the shape information are recognized through different streams includes:

[0009] Obtain training data; wherein, the training data includes color parameters and shape parameters;

[0010] Screen the training data according to preset requirements to determine the data type;

[0011] Classify the training data according to the data type to obtain a training set, a validation set, and a test set;

[0012] Train the neural network based on the training set, the validation set, and the test set.

[0013] Further, the step of training the neural network according to the training set, the validation set, and the test set includes:

[0014] Training the neural network according to the training set;

[0015] Calibrating the parameters of the trained neural network according to the validation set;

[0016] Evaluating the calibrated neural network according to the test set, and completing the training when the accuracy of the evaluation result is higher than a preset value.

[0017] Further, it further includes:

[0018] Obtaining a loss function, applying the loss function to the training set, the validation set, and the test set to obtain optimized setting parameters;

[0019] Optimizing the neural network according to the optimized setting parameters.

[0020] Further, the loss function is:

[0021]

[0022] where N is the number of samples; C is the number of categories; denotes the true value of the j-th label of the i-th sample, usually 0 or 1; denotes the predicted logit value of the j-th label of the i-th sample; is the sigmoid function applied to logitσ(x);

[0023] The calculation formula is

[0024]

[0025] Further, the weight of the difference in color information is the ratio of the Euclidean distance between the actual color and the target color to the maximum distance in the color space;

[0026] The weight of the difference in shape information is calculated by cosine similarity.

[0027] Further, the formula for the weight of the difference in color information is:

[0028]

[0029] D(C actual ,C target ) is the Euclidean distance between the actual color and the target color, and D Max is the maximum distance in the color space;

[0030]

[0031] Among them, C target =(R target , G target , B target ) is the target color, C actual =(R actual , G actual , B actual ) is the actual color, R target is the red component value of the target color, G target is the green component value of the target color, B target is the blue component value of the target color, R actual is the actually scanned red component value, G actual is the actually scanned green component value, B actual is the actually scanned blue component value;

[0032] The formula for the weight of the difference in the shape information is:

[0033] W shape =0.5×0.5(1 - Cosine Similarity(S actual , S target ));

[0034] Cosine Similarity(S actual , S target ) measures the similarity between shapes using cosine similarity for the shape information flow;

[0035]

[0036] A device for laser scanning and detecting complex graphics, comprising:

[0037] A training module for obtaining training data and training a neural network based on the training data so that the neural network has the ability to recognize color information and shape information; among them, the color information and the shape information are recognized through different flows;

[0038] An identification module for obtaining a target image and obtaining the target color information and target shape information of the target image based on the trained neural network;

[0039] A fusion module for feature - fusing the target color information and the target shape information to obtain the characteristic information of the target image.

[0040] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of any one of the above-mentioned methods for detecting complex graphics by laser scanning.

[0041] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned methods for detecting complex graphics by laser scanning are implemented.

[0042] This application has the following advantages:

[0043] In the embodiments of the present application, in response to the problems of insufficient precision, low accuracy, and poor detection speed in scanning complex images and diversified parts in the prior art, the present application provides a method for detecting complex graphics by laser scanning, including: obtaining training data, and training a neural network based on the training data, so that the neural network has the ability to recognize color information and shape information; wherein the color information and the shape information are recognized through different streams; obtaining a target image, and obtaining target color information and target shape information of the target image based on the trained neural network; performing feature fusion of the target color information and the target shape information to obtain characteristic information of the target image. The laser scanning detection method of the dual-stream model solves the problems of slow detection speed and insufficient precision in the prior art. The method analyzes the color and shape of the scanned object separately through the dual-stream model, which significantly improves the detection speed and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0045] Figure 1 This is a flowchart of the steps of a method for detecting complex graphics by laser scanning provided by an embodiment of the present application;

[0046] Figure 2 This is a schematic diagram of the module structure of a device for laser scanning and detecting complex graphics provided by an embodiment of the present application;

[0047] Figure 3 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the objects, features and advantages of the present application more obvious and understandable, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0049] The inventors analyzed the prior art and found that in the production of complex structural parts, the prior art uses manual or simple identification equipment for detection, and the detection precision is low and the detection accuracy is difficult to estimate, so that waste materials cannot be fed back to increase production costs and improve production efficiency.

[0050] Reference Figure 1 , shows a flowchart of the steps of a method for laser scanning to detect complex graphics in the present application, specifically:

[0051] S110, acquiring training data, and training a neural network according to the training data, so that the neural network has the ability to recognize color information and shape information; wherein the color information and the shape information are recognized through different streams;

[0052] S120, acquiring a target image, and obtaining target color information and target shape information of the target image according to the trained neural network;

[0053] S130 , performing feature fusion on the target color information and the target shape information to obtain characteristic information of the target image.

[0054] In the embodiments of the present application, in response to the problems of insufficient precision, low accuracy, and poor detection speed in scanning complex images and diversified parts in the prior art, the present application provides a method for detecting complex graphics by laser scanning, including: obtaining training data, and training a neural network based on the training data, so that the neural network has the ability to recognize color information and shape information; wherein the color information and the shape information are recognized through different streams; obtaining a target image, and obtaining target color information and target shape information of the target image based on the trained neural network; performing feature fusion of the target color information and the target shape information to obtain characteristic information of the target image. The laser scanning detection method of the dual-stream model solves the problems of slow detection speed and insufficient precision in the prior art. The method analyzes the color and shape of the scanned object separately through the dual-stream model, which significantly improves the detection speed and accuracy.

[0055] Next, a method and apparatus for detecting complex graphics by laser scanning in this exemplary embodiment will be further described.

[0056] As described in step S110 above, obtain training data and train a neural network based on the training data so that the neural network has the ability to recognize color information and shape information; wherein, the color information and the shape information are recognized by different streams.

[0057] It should be noted that the neural network is a two-stream AlexNet model. By virtue of the dual-stream feature of AlexNet, one stream is set to analyze the color details of the scanned item, and the other stream is used to analyze the shape parameters of the scanned item. Its features are as follows: stream1: a convolutional neural network for processing color information, stream2: a convolutional neural network for processing shape information, and the outputs of the two streams are fused, and the fused features are used for final classification or regression prediction.

[0058] In the AlexNet two-stream model, these two streams work independently and in parallel. They each receive input data and independently extract relevant features. The first stream focuses on extracting color information in the image. Using the input image of the RGB channel, after being processed by multiple convolutional layers, pooling layers, and activation functions, the color-related features in the image are gradually extracted; the second stream focuses on extracting the shape and structural features in the image, and performs similar steps as the first stream, but the difference is that the second stream is used to extract the shape-related features. After their respective convolutional processes, these two streams will perform feature fusion in the fully connected layer. The fused feature vector contains comprehensive information from both color and shape, thus forming a more comprehensive and accurate feature representation.

[0059] In an embodiment of the present invention, the specific process of "obtaining training data and training a neural network based on the training data so that the neural network has the ability to recognize color information and shape information; wherein, the color information and the shape information are recognized by different streams" described in step S110 can be further described in combination with the following description.

[0060] As described in the following steps, obtain training data; wherein, the training data includes color parameters and shape parameters.

[0061] It should be noted that by deploying a laser scanning device and its related sensors, laser scanning data including color and shape parameters is obtained, and the data is labeled to ensure that the color and shape information of each data point is accurate.

[0062] As described in the following steps, screen the training data according to preset requirements to determine the data type.

[0063] It should be noted that by determining the required data types and using an online information collection system, all the data types and their related information required by the system are determined, including part materials, colors, and shapes.

[0064] As described in the following steps, the training data is classified according to the data types to obtain a training set, a validation set, and a test set;

[0065] It should be noted that missing values and abnormal data are processed to ensure the quality of the data. The selected training data is standardized or normalized and divided into a training set, a validation set, and a test set. The training set is used for model training, generally accounting for 70%-80% of the total data. The validation set is used to adjust model parameters and select the best model, generally accounting for 10%-15% of the total data. The test set is used to evaluate the performance of the final model, generally accounting for 10%-15% of the total data.

[0066] As described in the following steps, the neural network is trained according to the training set, the validation set, and the test set.

[0067] The specific process of the step "training the neural network according to the training set, the validation set, and the test set" can be further described in combination with the following description.

[0068] As described in the following steps, the neural network is trained according to the training set;

[0069] As described in the following steps, the parameters of the trained neural network are corrected according to the validation set;

[0070] As described in the following steps, the corrected neural network is evaluated according to the test set. When the evaluation result accuracy rate is higher than the preset value, the training is completed.

[0071] It should be noted that three data sets are imported, the model is trained using the training set, the corresponding parameters of all parts are adjusted, the performance of the model is evaluated using the validation set, the final performance of the model is determined to maximize the accuracy rate, and finally the test set is used to test the accuracy of the model.

[0072] In an embodiment of the present application, it further includes:

[0073] As described in the following steps, a loss function is obtained, and the loss function is applied to the training set, the validation set, and the test set to obtain optimized setting parameters;

[0074] As described in the following steps, the neural network is optimized according to the optimized setting parameters.

[0075] It should be noted that when using the test set, the accuracy rate and parameters may not be accurate enough to meet the standards for being put into use. To ensure its accuracy and sustainable optimization, the loss function will be used to apply the three data sets through a function, compare the predicted values of all the data with the predicted values currently determined and applied by the model, and optimize the parameters until the optimal settings are found.

[0076] In an embodiment of the present application, the loss function is:

[0077]

[0078] where N is the number of samples; C is the number of categories; represents the true value of the j-th label of the i-th sample, usually 0 or 1; represents the predicted logit value of the j-th label of the i-th sample; is the sigmoid function applied to logitσ(x);

[0079] The calculation formula is

[0080]

[0081] It should be noted that the BCEwithLogits loss function combines the binary cross-entropy loss and the activation function, combines the S-type layer and the binary cross-entropy loss in one formula, simplifies the derivative calculation in the optimization process. The role of σ(x) is to convert the logit value into a probability value. The purpose of the BCEWithLogits loss function is to measure the difference between the predicted probability and the true label. For each sample and each label, the matching degree between the prediction and the true value is calculated.

[0082] In an embodiment of the present application, the weight of the difference in color information is the ratio of the Euclidean distance between the actual color and the target color to the maximum distance in the color space;

[0083] The weight of the difference in shape information is calculated by cosine similarity.

[0084] It should be noted that the color information stream uses the RGB model to extract the color-related features in the image. The RGB model defines any color by specifying the intensities of three primary colors: red (R), green (G), and blue (B). The value of each color component is usually between 0 and 255 (in 8-bit representation), where: R represents the intensity of the red component, G represents the intensity of the green component, and B represents the intensity of the blue component.

[0085] In an embodiment of the present application, the formula for the weight of the difference in color information is:

[0086]

[0087] D(C actual ,C target ) is the Euclidean distance between the actual color and the target color, and D Max is the maximum distance in the color space;

[0088]

[0089] Among them, C target = (R target , G target , B target ) is the target color, and C actual = (R actual , G actual , B actual ) is the actual color, R target is the red component value of the target color, G target is the green component value of the target color, B target is the blue component value of the target color, R actual is the actually scanned red component value, G actual is the actually scanned green component value, B actual is the actually scanned blue component value;

[0090] The formula for the weight of the difference in the shape information is:

[0091] W shape = 0.5×0.5(1 - Cosine Similarity(S actual , S target ));

[0092] Cosine Similarity(S actual , S target ) measures the similarity between shapes using cosine similarity for the shape information flow;

[0093]

[0094] It should be noted that the specific information of the expression of the color weight is: W color is the color weight, D(C actual , C target ) is the Euclidean distance between the actual color and the target color, and D Max is the maximum possible distance in the color space (from (0, 0, 0) to (255, 255, 255). This formula ensures that when the color difference is the largest (such as extreme cases like black and white), the weight will approach 0, and when the colors match exactly, the weight is equal to 0.5. If the result is too different from 0.5, the color range will not pass the review;

[0095] The shape information flow measures the similarity between shapes using cosine similarity. The shapes are represented by vectors, and its specific expression is as follows:

[0096]

[0097] This value is between -1 and 1: when the cosine similarity is equal to 1, it means the shapes are exactly the same; when the cosine similarity is equal to -1, it means the shapes are exactly opposite; when the cosine similarity is equal to 0, it means the shapes have no correlation.

[0098] When the shapes match exactly, the cosine similarity is 1, and at this time, Wshape is 0.5; if the shapes are quite different, the cosine similarity value is close to 0 or negative, and Wshape will be far from 0.5.

[0099] After the color information flow and the data information flow process their respective part information respectively, feature information fusion is performed in the fully connected layer. The fused feature vector contains comprehensive information from color and shape, thus forming a more comprehensive and accurate feature representation, which is defined as the overall weight W total , and its expression is:

[0100] W total = W color + W shape

[0101] If W total is 1, it is completely qualified. If W color or W shape is not 0.5 for either of them, it means the model has defects in terms of color and shape. The two-stream model of AlexNet is used to simultaneously analyze the color details and shape parameters of the scanned object. One stream focuses on processing color information, and the other stream focuses on shape information, so that defects in parts can be quickly and accurately identified during the detection process.

[0102] As described in step S120 above, obtain the target image, and obtain the target color information and target shape information of the target image according to the trained neural network.

[0103] As described in step S130 above, perform feature fusion on the target color information and the target shape information to obtain the characteristic information of the target image.

[0104] In a specific implementation, the parts produced by a production line are detected by the method proposed in this solution, and the steps are as follows:

[0105] Data collection: Generate a database containing part images, import 1000 images, each with a size of 256x256 pixels. The dataset contains 3 different colors and 16 different shapes, and each image is labeled with shape and color information. We divide these images into two parts: the shape data stream and the color data stream.

[0106] Model design and training: Use the two streams of the method proposed in this application for analysis. For the first stream, it is passed to this part through the color channel of the input image; for the second stream, the input image is morphologically processed to highlight the shape features and then passed to this part;

[0107] Train colors and shapes, and conduct unified analysis and learning of 700 samples for colors and shapes. The model simulates the part features under multiple color and shape combinations;

[0108] By using the BCEWithLogits loss function, the consistency between the model prediction and the actual label is maximized through this loss function, reducing classification errors. Through this loss function, the model can effectively distinguish different color and shape combinations, thereby improving the detection accuracy.

[0109] After importing the sample model, the model will analyze 150 sample images. The prediction parameters obtained through the loss function are compared with the scanning parameters of this image sample, and through the least squares method inside the model, the best fitting parameter combination of these 150 image samples is extracted. The system parameters are slightly modified to minimize the difference between the model output and the actual detection result, calibrating the model parameters to optimize its performance.

[0110] Import the remaining 150 image samples in the database and use the parameter combination in the database for detection to ensure that the accuracy rate on the test set reaches 95% or above. If the accuracy rate cannot reach more than 95%, the validation set will randomly extract 150 image samples from the training set again, and the model will be retrained, validated, and tested on the image samples until the accuracy rate reaches more than 95%. That is, after the model can accurately detect the color and shape anomalies of the parts, the model is put into use.

[0111] During the detection process, each part image scanned on the production line will be input into the trained two-stream model. The model will analyze the color and shape features respectively. If the detection accuracy rate of this scan round is found to be higher than the accuracy rate of the currently used parameter combination through the loss function, the model will automatically update the parameter combination, that is, the model self-optimizes.

[0112] During the process of being put into use, a total of 1,000 parts are scanned in one training round. In this training round, the model scans a total of 32 defective parts. After manual inspection, there are 35 defective parts in this training round. Among them, the model misses 4 defective parts. One part has a color difference in the welding area that is too large compared to the normal parts, resulting in a misdetection. The number of non-defective parts = total number of parts - actual number of defective parts = 1,000 - 35 = 965. The number of correctly identified non-defective parts = actual number of correctly detected defective parts + number of correctly identified non-defective parts - number of misdetected parts by the model = 27 + 964 = 990. The overall accuracy rate is: (total number of correctly detected by the model) / (total number of scanned parts) = 990 / 1,000 = 99%. It meets the actual use requirements.

[0113] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For related parts, please refer to the partial description of the method embodiment.

[0114] Refer to Figure 2 , which shows a device for laser scanning and detecting complex graphics provided by an embodiment of the present application;

[0115] Specifically, it includes:

[0116] A training module 210, configured to obtain training data and train a neural network based on the training data, so that the neural network has the ability to recognize color information and shape information; wherein, the color information and the shape information are recognized through different streams;

[0117] An identification module 220, configured to obtain a target image and obtain the target color information and target shape information of the target image based on the trained neural network;

[0118] A fusion module 230, configured to perform feature fusion on the target color information and the target shape information to obtain the characteristic information of the target image.

[0119] In an embodiment of the present invention, the training module 210 includes:

[0120] An acquisition unit, configured to obtain training data; wherein, the training data includes color parameters and shape parameters;

[0121] A screening unit, configured to screen the training data according to preset requirements to determine the data type;

[0122] A classification unit, configured to classify the training data according to the data type to obtain a training set, a validation set, and a test set;

[0123] A training unit, configured to train the neural network based on the training set, the validation set, and the test set.

[0124] In an embodiment of the present invention, the training unit includes:

[0125] A pre-training sub-unit for training the neural network according to the training set;

[0126] A correction sub-unit for correcting the parameters of the trained neural network according to the validation set;

[0127] An evaluation sub-unit for evaluating the corrected neural network according to the test set. When the accuracy rate of the evaluation result is higher than the preset value, the training is completed.

[0128] In an embodiment of the present invention, it further includes:

[0129] A loss function unit for obtaining a loss function, and applying the loss function to the training set, the validation set, and the test set to obtain optimized setting parameters;

[0130] An optimization unit for optimizing the neural network according to the optimized setting parameters.

[0131] In an embodiment of the present invention, the loss function unit includes:

[0132] A loss function setting sub-unit for setting the loss function as

[0133]

[0134] where N is the number of samples; C is the number of categories; represents the true value of the j-th label of the i-th sample, usually 0 or 1; represents the predicted logit value of the j-th label of the i-th sample; is the sigmoid function applied to logitσ(x);

[0135] The calculation formula is

[0136]

[0137] In an embodiment of the present invention, it further includes:

[0138] A color weight unit for calculating color weights. The weight of the difference in color information is the ratio of the Euclidean distance between the actual color and the target color to the maximum distance in the color space;

[0139] A shape weight unit for calculating shape weights. The weight of the difference in shape information is calculated by cosine similarity.

[0140] In an embodiment of the present invention, it further includes:

[0141] A formula setting unit for setting a weight formula:

[0142] The formula for the weight of the difference in the color information is:

[0143]

[0144] D(C actual , C target ) is the Euclidean distance between the actual color and the target color, and D Max is the maximum distance in the color space;

[0145]

[0146] Where C target =(R target , G target , B target ) is the target color, C actual =(R actual , G actual , B actual ) is the actual color, R target is the red component value of the target color, G target is the green component value of the target color, B target is the blue component value of the target color, R actual is the actually scanned red component value, G actual is the actually scanned green component value, B actual is the actually scanned blue component value;

[0147] The formula for the weight of the difference in the shape information is:

[0148] W shape =0.5×0.5(1 - Cosine Similarity(S actual , S target ));

[0149] Cosine Similarity(S actual , S target ) measures the similarity between shapes using cosine similarity for the shape information flow;

[0150]

[0151] Referring to Figure 3 , a computer device for a method of laser scanning and detecting complex graphics according to the present invention is shown, and specifically may include the following:

[0152] The computer device 12 described above is presented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects different system components (including the system memory 28 and the processing unit 16).

[0153] The bus 18 represents one or more of several types of bus 18 architectures, including a memory bus 18 or a memory controller, a peripheral bus 18, a graphics acceleration port, a processor, or a local bus 18 using any of the various bus 18 architectures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus 18, Micro Channel Architecture (MAC) bus 18, Enhanced ISA bus 18, Video Electronics Standards Association (VESA) local bus 18, and Peripheral Component Interconnect (PCI) bus 18.

[0154] The computer device 12 typically includes a variety of computer system-readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0155] The system memory 28 may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 34 can be used for reading and writing on non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). Although Figure 3 not shown, a disk drive for reading and writing on a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing on a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM, or other optical media) can be provided. In these cases, each drive can be connected to the bus 18 through one or more data media interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 42, and these program modules 42 are configured to perform the functions of the various embodiments of the present invention.

[0156] A program / utility 40 having a set (at least one) of program modules 42 can be stored, for example, in the memory. Such program modules 42 include - but are not limited to - an operating system, one or more application programs, other program modules 42, and program data. Each or some combination of these examples may include the implementation of a network environment. The program modules 42 generally perform the functions and / or methods described in the embodiments of the present invention.

[0157] The computer device 12 can also communicate with one or more external devices 14 (such as keyboards, pointing devices, displays 24, cameras, etc.), and can also communicate with one or more devices that enable an operator to interact with the computer device 12, and / or communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (such as network cards, modems, etc.). Such communication can be carried out through the input / output (I / O) interface 22. Moreover, the computer device 12 can also communicate with one or more networks (such as local area networks (LANs)), wide area networks (WANs), and / or public networks (such as the Internet) through the network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the computer device 12 through the bus 18. It should be understood that although Figure 3 not shown in the figure, other hardware and / or software modules can be used in combination with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units 16, external disk drive arrays, RAID systems, tape drives, and data backup storage systems 34, etc.

[0158] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a method for laser scanning and detecting complex graphics provided by an embodiment of the present invention.

[0159] That is, when the above-mentioned processing unit 16 executes the above program, it realizes: obtaining training data, and training a neural network based on the training data so that the neural network has the ability to recognize color information and shape information; wherein, the color information and the shape information are recognized through different streams;

[0160] Obtaining a target image, and obtaining the target color information and target shape information of the target image based on the trained neural network;

[0161] Performing feature fusion on the target color information and the target shape information to obtain the characteristic information of the target image.

[0162] In an embodiment of the present invention, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it realizes a method for laser scanning and detecting complex graphics provided by all embodiments of the present application:

[0163] That is, when the program is executed by a processor, it realizes: obtaining training data, and training a neural network based on the training data so that the neural network has the ability to recognize color information and shape information; wherein, the color information and the shape information are recognized through different streams;

[0164] Obtain a target image, and obtain the target color information and target shape information of the target image according to the trained neural network;

[0165] Perform feature fusion on the target color information and the target shape information to obtain the characteristic information of the target image.

[0166] Any combination of one or more computer-readable media may be employed. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPOM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0167] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take many forms, including - but not limited to - an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0168] Computer program code for performing the operations of the present invention may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the healthcare provider's computer, partially on the healthcare provider's computer, executed as a stand-alone software package, partially on the healthcare provider's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the healthcare provider's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference may be made to each other.

[0169] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0170] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or terminal device comprising the element.

[0171] The above provides a detailed introduction to a method and device for laser scanning and detecting complex graphics provided by the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for detecting complex graphics by laser scanning, characterized in that: include: Acquire training data, and train a neural network based on the training data, so that the neural network has the ability to recognize color information and shape information; wherein the color information and the shape information are recognized through different streams; Acquire a target image, and obtain target color information and target shape information of the target image according to the trained neural network; The target color information and the target shape information are feature-fused to obtain characteristic information of the target image.

2. The method according to claim 1, characterized in that The step of obtaining training data and training a neural network according to the training data so that the neural network has the ability to recognize color information and shape information; wherein the step of recognizing the color information and the shape information through different streams includes: Acquire training data; wherein the training data includes color parameters and shape parameters; Screening the training data according to preset requirements to determine the data type; Classifying the training data according to the data type to obtain a training set, a validation set, and a test set; The neural network is trained according to the training set, the validation set and the test set.

3. The method according to claim 2, characterized in that The step of training the neural network based on the training set, the validation set and the test set comprises: Training the neural network according to the training set; Correcting parameters of the trained neural network according to the verification set; The corrected neural network is evaluated based on the test set, and the training is completed when the accuracy of the evaluation result is higher than a preset value.

4. The method according to claim 3, characterized in that Also includes: Obtaining a loss function, and applying the loss function to the training set, the validation set, and the test set to obtain optimized setting parameters; The neural network is optimized according to the optimization setting parameters.

5. The method according to claim 4, characterized in that The loss function is: Where N is the number of samples; C is the number of categories; The true value of the jth label of the i-th sample is usually 0 or 1; Represents the predicted logit value of the jth label of the i-th sample; is the sigmoid function applied to logitσ(x); The calculation formula is 6. The method according to claim 1, characterized in that The weight of the difference in color information is the ratio of the Euclidean distance between the actual color and the target color to the maximum distance in the color space; The weight of the difference in shape information is calculated by cosine similarity.

7. The method according to claim 6, characterized in that The formula for the weight of the difference in color information is: D(C actual ,C target ) is the Euclidean distance between the actual color and the target color, D Max is the maximum distance in the color space; Among them, C target =(R target ,G target ,B target ) is the target color, C actual =(R actual ,G actual ,B actual ) is the actual color, R target is the red component value of the target color, G target is the green component value of the target color, B target is the blue component value of the target color, R actual is the actual scanned red component value, G actual is the green component value actually scanned, B actual is the actual scanned blue component value; The formula for the weight of the difference in shape information is: W shape =0.5×0.5(1-Cosine Similarity(S actual ,S target )); Cosine Similarity(S actual ,S target ) uses cosine similarity to measure the similarity between shapes for shape information flow; 8. A device for laser scanning and detecting complex graphics, characterized in that: include: A training module, used to obtain training data, and train a neural network according to the training data, so that the neural network has the ability to recognize color information and shape information; wherein the color information and the shape information are recognized through different streams; A recognition module, used to acquire a target image, and obtain target color information and target shape information of the target image according to the trained neural network; The fusion module is used to perform feature fusion on the target color information and the target shape information to obtain characteristic information of the target image.

9. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the method for detecting complex graphics by laser scanning as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for detecting complex graphics by laser scanning as described in any one of claims 1 to 7 are implemented.

Images