A product quality identification device and method

Abstract

The present application provides a product quality identification device and method, which can be used in the field of product quality monitoring or other fields. The device includes: a shooting table for setting the product to be identified; a bracket arranged on the shooting table; multiple cameras arranged on the shooting table or the bracket through universal magic hands and shooting from multiple different angles; a switch connected to the multiple cameras and the server through network cables respectively, and used for transmitting multiple product pictures taken by the multiple cameras to the server; and a server for respectively extracting the detailed feature images and semantic feature images of each product picture, and performing attention distribution and multiple fusion operations to determine the quality score corresponding to the product. The quality identification device and method provided by the present application realize the determination of the final quality score of the product to be identified. During the process of determining the quality score, the influence of factors such as light and shooting angle on picture recognition is eliminated, and through the setting of the switch, the offline identification of product quality is realized.

Description

Technical Field

[0001] This application relates to the technical field of product quality monitoring, and particularly to a product quality identification device and method. Background Art

[0002] In commodity sales management, the quality of commodities is often the key content of monitoring and management. Especially for fresh products, during the circulation process, the products will gradually decay and deteriorate over time. Therefore, manufacturers and distributors need to strictly control the quality.

[0003] In the prior art, there are many devices for identifying products such as flowers. These devices mainly identify through apps or mini-programs, such as Baidu Image Recognition, WeChat Scan Recognition, etc. These devices can often only identify the variety of products, cannot grade the product quality, and the identification process needs to rely on network links to complete, making it difficult to identify and extract product information in an offline state. In addition, existing product identification devices are often handheld for shooting. However, due to different factors such as the angle and light of the captured pictures, the main body orientations in the product pictures are diverse, resulting in a low identification accuracy problem. Sometimes the accuracy is even lower than 40%. Summary of the Invention

[0004] Aiming at the problems in the prior art, this application provides a product quality identification device and method, which can at least partially solve the problems existing in the prior art.

[0005] In a first aspect, this application provides a product quality identification device, including: a shooting table, a bracket, a universal magic arm, multiple cameras, a switch, and a server;

[0006] The bracket is arranged on the shooting table, and the shooting table is used to place the product to be identified;

[0007] The multiple cameras are respectively arranged on the shooting table or the bracket through the universal magic arm, and shoot from multiple different angles;

[0008] The switch is connected to the multiple cameras and the server through network cables respectively, and is used to transmit multiple product pictures captured by the multiple cameras to the server;

[0009] The server is used to respectively extract the detailed feature image and semantic feature image of each product picture, perform attention allocation and multiple fusion operations on the detailed feature image and semantic feature image to obtain a product quality feature image, determine the quality score corresponding to each product picture according to the product quality feature image; determine the final quality score of the product to be identified according to the product quality corresponding to each product picture.

[0010] Among them, the multiple cameras respectively take pictures from the horizontal angle, the vertical top-down angle, and the inclined angle.

[0011] Among them, it further includes: a camera control device, which is respectively connected to the multiple cameras and is used to control the multiple cameras to take pictures.

[0012] Among them, the product quality identification device further includes: a barcode reader, which is connected to the switch through a network cable and is used to read the identification code on the product to be identified.

[0013] Among them, the server includes:

[0014] A detailed feature image extraction unit, which is used to input the received product picture into a detailed feature extraction model to obtain a detailed feature image;

[0015] A semantic feature image extraction unit, which is used to input the received product picture into a semantic feature extraction model to obtain a semantic feature image;

[0016] An attention allocation unit, which is used to input the detailed feature image and the semantic feature image into an attention allocation model respectively to obtain a detailed attention feature image and a semantic attention feature image;

[0017] An image fusion unit, which is used to perform multiple fusions on the detailed feature image, the detailed attention feature image, the semantic feature image, and the semantic attention feature image to obtain a quality feature image;

[0018] A picture quality score determination unit, which is used to obtain the quality score corresponding to the product picture according to the quality feature image.

[0019] Among them, the image fusion unit includes:

[0020] A first fusion module, which is used to fuse the detailed feature image and the detailed attention feature image to obtain a fused detailed feature image;

[0021] A second fusion module, which is used to fuse the semantic feature image and the semantic attention feature image to obtain a fused semantic feature image;

[0022] A third fusion module, which is used to fuse the fused detailed feature image and the fused semantic feature image to obtain the quality feature image.

[0023] Among them, the server further includes:

[0024] A final quality score determination unit, which is used to determine the final quality score of the product to be identified according to the quality scores corresponding to each product picture and a preset quality score fitting model.

[0025] Among them, the server further includes: a qualified rate calculation module, which calculates the qualified rate of the products according to each recognition result and the lowest product quality standard when the recognition of a preset number of products is completed.

[0026] Among them, it further includes: a power supply, connected to the multiple cameras.

[0027] In a second aspect, the present application provides a product quality recognition method, which is implemented based on the product quality recognition device described in any of the above embodiments, and includes:

[0028] Obtain the number of the product to be recognized and input it into the server;

[0029] Turn on the multiple cameras and obtain multiple pictures of the product to be recognized from different angles;

[0030] The switch transmits the multiple pictures collected by the cameras to the server;

[0031] The server stores the multiple pictures into the document corresponding to the number, and respectively extracts the detailed feature image and semantic feature image of each product picture;

[0032] Perform attention allocation and multiple fusion operations on the detailed feature image and semantic feature image to obtain a product quality feature image, and determine the quality score corresponding to each product picture according to the product quality feature image;

[0033] Determine the final quality score of the product to be recognized according to the product quality corresponding to each product picture.

[0034] Among them, the step of obtaining the number of the product to be recognized and inputting it into the server includes:

[0035] The barcode reader recognizes the identification code on the product to be recognized to obtain the code corresponding to the product to be recognized;

[0036] The switch transmits the code recognized by the barcode reader to the server.

[0037] Among them, it further includes:

[0038] When the recognition of a preset number of products is completed, the server calculates the qualified rate of the products according to each recognition result and the lowest product quality standard.

[0039] The product quality identification device and method provided by this application are configured with a shooting table, a bracket, a universal magic arm, multiple cameras, a switch, and a server. Among them, the shooting table is used to place the product to be identified, the bracket is set on the shooting table, and multiple cameras are respectively set on the shooting table or the bracket through the universal magic arm for shooting from multiple different angles. The switch is connected to multiple cameras and the server through network cables respectively, and is used to transmit multiple product pictures taken by the multiple cameras to the server. The server is used to extract the detailed feature images and semantic feature images of each product picture respectively, and further perform attention allocation and multiple fusion operations to obtain the product quality feature image, and determine the quality score corresponding to each product picture according to the product quality feature image, thus realizing the determination of the final quality score of the product to be identified. During the process of determining the quality score, the influence of factors such as light and shooting angle on picture recognition is eliminated, and by setting the switch and making the recognition algorithm in the form of an executable file, the offline identification of product quality is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0041] Figure 1 It is a schematic structural diagram of a product quality identification device provided by an embodiment of this application;

[0042] Figure 2 It is a schematic structural diagram of a product quality identification device provided by an embodiment of this application;

[0043] Figure 3 It is a schematic structural diagram of the universal magic arm provided by an embodiment of this application;

[0044] Figure 4 It is a schematic structural diagram of the server provided by an embodiment of this application;

[0045] Figure 5 It is a schematic structural diagram of a detailed feature extraction model provided by some embodiments of this application;

[0046] Figure 6 It is a schematic structural diagram of a semantic feature extraction model provided by some embodiments of this application;

[0047] Figure 7 It is a schematic structural diagram of an attention allocation model provided by some embodiments of this application;

[0048] Figure 8It is a schematic structural diagram of an image fusion unit provided by some embodiments of the present application;

[0049] Figure 9 It is a schematic structural diagram of a server provided by an embodiment of the present application;

[0050] Figure 10 It is a schematic structural diagram of a server provided by an embodiment of the present application;

[0051] Figure 11 It is a flowchart of a product quality identification method provided by some embodiments of the present application;

[0052] Figure 12 It is a flowchart of a product quality identification method provided by some embodiments of the present application. Detailed implementation manners

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer and more understandable, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Herein, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but not to limit the present invention. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other.

[0054] Figure 1 It is a schematic structural diagram of a product quality identification device provided by an embodiment of the present application. As Figure 1 shown, the product quality identification device provided by the present application includes: a shooting table 110, a bracket 120, a universal magic hand 130, multiple cameras 140, a switch 150, and a server 160;

[0055] Among them, the bracket 120 is arranged on the shooting table 110, and the shooting table 110 is used to place the product to be identified;

[0056] Specifically, as Figure 1 shown, two mutually perpendicular scales 111 can be arranged on the surface of the shooting table 110, so that with the scales 111 as a reference, the product to be identified can be placed in a suitable position. The bracket 120 is used to provide support for the camera 140, and can be composed of multiple bracket rods or bracket plates. When the bracket 120 includes a bracket plate perpendicular to the shooting table 110, a vertical scale 121 can also be drawn on the bracket plate according to needs, so as to further determine the approximate height of the product to be identified.

[0057] In one embodiment, the center of the bottom surface of the product to be recognized can be made to coincide with the intersection point of the two scales 111; in another embodiment, one side of the product to be recognized can also be made to coincide with or be tangent to one of the scales. The present application does not limit the specific placement method of the product to be recognized. In addition, the number of markings, the angles between the scales, and the relative position relationship can also be adjusted according to actual needs.

[0058] Multiple cameras 140 are respectively arranged on the shooting table 110 or the bracket 120 through universal magic hands 130 and are shot from multiple different angles;

[0059] Specifically, one end of the universal magic hand 130 is fixed on the shooting table 110 or the bracket 120, the other end is provided with a camera 140, and a rotatable joint is arranged between the two ends, so as to realize the adjustment of the angle of the camera 140. By adjusting the universal magic hand 130, multiple cameras 140 are fixed at specific angles, so as to prevent the difference in shooting angles from affecting the product recognition result during each shooting. In addition, multiple specific angles can be set so that at least two cameras 140 correspond to different shooting angles, so as to obtain product pictures from multiple different angles and prevent inaccurate product quality recognition caused by some features of the product being blocked when the product quality is recognized from a single angle.

[0060] In the laboratory environment, the height, openness standard, and environmental brightness of the product can be unified; in the actual environment, the focal length of the camera and the environmental brightness can be adjusted according to actual needs.

[0061] In one embodiment, as Figure 2 shown, among the multiple cameras 140, there may be a horizontal angle camera 141 (front view parallel to the ground at 0°), a vertical downward view camera 142 (vertical downward view of 180° to the ground), and an inclined angle camera 143 (for example, shooting obliquely at 45° to the ground), but the present application is not limited thereto.

[0062] The camera parameters of the camera 140 can be set according to the actual situation, for example: resolution: 2592*1944; target surface: 1 / 2.5 CMOS; color; frame rate: 15fps; line exposure; lens parameters: focal length: 6mm; five million pixels, but the present application is not limited thereto.

[0063] In one embodiment, as Figure 3 shown, the universal magic hand 130 includes: a base 131, a first clamping piece 132, a first tightening nut 133, a first bolt (not shown), a first rotating arm 134, a second clamping piece 135, a second tightening nut 136, a second bolt 137, a second rotating arm 138, and a camera clamping part 139;

[0064] Among them, the base 131 can be fixed on the bracket 120 or the shooting table 110. At the top of the base 131, two first clamping pieces 132 are fixedly connected. The first rotating arm 134 is arranged between the two first clamping pieces 132. The first bolt passes through the two first clamping pieces 132 and the first rotating arm 134. Two first tightening nuts 133 are sleeved on the first bolt and are distributed on both sides of the two first clamping pieces 132 and the first rotating arm 134. When the two first tightening nuts 133 are tightened inward, the two first clamping pieces 132 tightly clamp the first rotating arm 134, so that the angle of the first rotating arm 134 is fixed. After loosening any one of the first tightening nuts 133, the first rotating arm 134 can rotate between the two first clamping pieces 132 with the first bolt as the axis.

[0065] At one end of the first rotating arm 134 away from the first clamping piece 132, two second clamping pieces 135 are arranged. The second rotating arm 138 is arranged between the two second clamping pieces 135. The second bolt 137 passes through the two second clamping pieces 135 and the second rotating arm 138. Two second tightening nuts 136 are sleeved on the second bolt 137 and are respectively arranged on both sides of the two second clamping pieces 135 and the second rotating arm 138. Similarly, when the two second tightening nuts 136 are tightened inward, the two second clamping pieces 135 tightly clamp the second rotating arm 138, so that the angle of the second rotating arm 138 is fixed. After loosening any one of the second tightening nuts 136, the second rotating arm 138 can rotate between the two second clamping pieces 135 with the second bolt 137 as the axis.

[0066] Threads can be provided only at both ends of the first bolt and the second bolt, and the middle part thereof can be set as a smooth surface, so that while the first tightening nut 133 and the second tightening nut 136 can be tightened and fixed, the first rotating arm 134 and the second rotating arm 138 can rotate freely. The first rotating arm 134 can rotate in a plane parallel to the paper surface, and the second rotating arm 138 can rotate in a plane perpendicular to the paper surface. Through the cooperation of the first rotating arm 134 and the second rotating arm 138, arbitrary setting of the angle of the camera 140 is achieved.

[0067] In addition, the multi-angle rotation of the rotating arm can also be realized by a rotatable ball head. One end of the universal magic hand 130 can also be a clamping part (not shown), and it is clamped on the bracket or the shooting table through the clamping part. Any bracket that can rotate at multiple angles can be used as the universal magic hand 130 in this application, and the specific structure of the universal magic hand 130 is not limited in this application.

[0068] In one embodiment, as Figure 1 shown, the product quality identification device provided by this application further includes: a camera control device 170, which is respectively connected to multiple cameras 140 and is used to control the multiple cameras 140 to take pictures.

[0069] Specifically, the camera control device 170 can be, for example, a tablet computer that externally connects multiple cameras 140, a mobile device installed with the corresponding control software of the camera 140, a control device provided by the camera manufacturer, etc. The present application does not limit the specific device types of the camera control device 170.

[0070] By setting the camera control device 170, it is possible to control each camera 140 to take pictures simultaneously with one key, saving manpower and further reducing the interference of other irrelevant factors. In addition, it is also possible to manually control each camera 140 to take pictures, but the present application does not take this as a preferred embodiment.

[0071] The switch 150 is connected to multiple cameras 140 and the server 160 respectively through network cables, and is used to transmit multiple product pictures taken by the multiple cameras 140 to the server 160;

[0072] Specifically, the interfaces of the server 160 are often limited. By setting the switch 150 to be connected to each camera 140 through multiple interfaces, and then transmitting the pictures input by each camera 140 to the server 160 centrally through a single interface, the problem of insufficient server interfaces is solved.

[0073] The subsequent algorithm for processing product pictures can be stored in the server 160 in the form of an executable file. The switch 150 is connected to multiple cameras 140 and the server 160 respectively through network cables, so that even in the case of poor network signals or even no network signals, the product pictures can be transmitted to the server 160 through the network cables in a wired manner, and the product pictures can be processed by the executable file in the server 160, overcoming the problem that product recognition cannot be achieved in the case of no network signals.

[0074] In addition, under the condition of the existence of a network, the server 160 can further upload the product pictures to an online server such as a cloud platform for further storage and processing.

[0075] The server 160 is used to extract the detailed feature image and semantic feature image of each product picture respectively, perform attention allocation and multiple fusion operations on the detailed feature image and semantic feature image to obtain the product quality feature image, and determine the quality score corresponding to each product picture according to the product quality feature image; determine the final quality score of the product to be recognized according to the product quality corresponding to each product picture.

[0076] Specifically, as Figure 4 shown, the server 160 includes: a detailed feature image extraction unit 161, a semantic feature image extraction unit 162, an attention allocation unit 163, an image fusion unit 164, and a picture quality score determination unit 165;

[0077] Among them, the detailed feature image extraction unit 161 is used to input the received product picture into the detailed feature extraction model to obtain the detailed feature image;

[0078] Specifically, Figure 5 is a schematic structural diagram of the detailed feature extraction model provided by some embodiments of the present application. As Figure 5 shown, the detailed feature extraction model includes several sequentially connected convolution modules 510. Each convolution module includes a convolution layer, and a normalization layer, a pooling layer, etc. can also be selectively set in each convolution layer as needed, and ReLU is used as the activation function of each convolution layer.

[0079] When performing detailed feature extraction, the product picture is input into the first convolution module. Except for the first convolution module, the input of each convolution module is the output of the previous convolution module, and the feature image output by the last convolution module is the extracted detailed feature image. The number of convolution modules can be set according to the actual situation, for example, it is 5, but the present application is not limited thereto. However, the detailed features will be lost as the neural network deepens. Therefore, the detailed feature extraction model should not include too many convolution modules, for example, it should not exceed the first preset number, and the first preset number can be determined through multiple experiments, and the present application does not limit this.

[0080] In addition, before inputting the feature image output by the previous convolution module into the next convolution module, the output feature image can be segmented by channel according to the size of the output feature image and the bearing capacity of a single GPU, and the next convolution module can be deployed on multiple GPUs at the same time. The segmented feature images of the previous convolution module are respectively input into the next convolution module on different GPUs for the next step of feature extraction.

[0081] It should be noted that splitting the feature image for training may cause the loss of the correlation relationship between some channels. Therefore, the feature image should not always be scattered to different GPUs for training. After the segmented feature image is extracted by the second preset number of convolution modules, the feature images extracted on each GPU can be re - stitched and further feature extraction can be performed.

[0082] The semantic feature image extraction unit 162 is used to input the received product picture into the semantic feature extraction model to obtain the semantic feature image;

[0083] Specifically, Figure 6 is a schematic structural diagram of the semantic feature extraction model provided by some embodiments of the present application. As Figure 6As shown, the semantic feature extraction model includes a backbone network 610 and a semantic feature extraction module 620. Among them, the backbone network 610 is formed by sequentially connecting multiple combined convolution modules (Darknetconv2d_BN_Leaky, DBL). Each combined convolution module includes a convolutional layer, a normalization layer (BatchNormalization, BN), and a ReLU activation function. In the backbone network 610, the role of the pooling layer is replaced by setting the stride for the convolutional layer in each combined convolution module, and only a max pooling layer (Maxpooling) is set after the first combined convolution module, thereby reducing the computational amount. In one embodiment, the semantic feature extraction module 620 may include a combined convolution module, a convolutional layer (Conv), and a max pooling layer (Maxpooling) connected in sequence, but the present application is not limited thereto, and the specific structure of the semantic feature extraction model can be appropriately adjusted according to actual needs.

[0084] The product image is input into the backbone network 610. After being extracted by the third preset number of DBL modules, a shallow semantic feature image is obtained. Then, the shallow semantic feature image is continuously extracted. After passing through the fourth preset number of DBL modules, a deep semantic feature image is obtained. Then, the shallow semantic feature image and the deep semantic feature image obtained through the backbone network 610 are concatenated. Among them, the shallow semantic feature image and the deep semantic feature image have the same length and width, and the number of channels of the concatenated image is equal to the sum of the number of channels of the shallow semantic feature image and the deep semantic feature image. The concatenated image is input into the semantic feature extraction module 620 to extract the final semantic feature image. The third preset number can be, for example, 25, and the fourth preset number can be, for example, 18, but the present application is not limited thereto.

[0085] Compared with the deep semantic feature image, the shallow semantic feature image has richer position feature information, while the further extracted deep semantic feature image contains more advanced semantic information. By further extracting features from the shallow semantic feature image and the fused deep semantic feature image, it is possible to retain more position feature information while obtaining high-level semantic features, and at the same time, it can further prevent overfitting and improve the accuracy of quality score evaluation.

[0086] The attention allocation unit 163 is used to input the detail feature image and the semantic feature image into the attention allocation model respectively to obtain a detail attention feature image and a semantic attention feature image;

[0087] Specifically, Figure 7 is a schematic structural diagram of the attention allocation model provided by some embodiments of the present application, as Figure 7As shown in the figure, the attention allocation model includes a first attention allocation channel 710 and a second attention allocation channel 720.

[0088] In the first attention allocation channel 710, an average pooling layer (AVGpooling) is used to compress the input image so as to compress each channel to a size of 1×1. After that, two fully connected layers are set to further extract the pixel values corresponding to the compressed channels to obtain the first weight feature. Among them, after the output neurons of the first fully connected layer, the ReLU activation function is used to activate the output data. The output neurons of the first fully connected layer are the input neurons of the second fully connected layer. The number of input neurons of the first fully connected layer and the number of output neurons of the second fully connected layer are equal to the number of channels of the input image. In one embodiment, the number of input neurons of the first fully connected layer can be represented by C, then the number of output neurons of the first fully connected layer can be represented by C×r, where r is a constant greater than 0 and less than or equal to 1. By setting the scaling parameter r, the number of neurons in the fully connected layer can be reduced, thereby further reducing the computational amount.

[0089] The structure of the second attention allocation channel 720 is basically the same as that of the first attention allocation channel 710. The difference is that in the second attention allocation channel 720, a max pooling layer (Maxpooling) is used to compress the input image. For subsequent operations, reference can be made to the description of the first attention allocation channel 710, and finally the second weight feature is obtained. The repeated parts will not be elaborated.

[0090] After obtaining the first weight feature and the second weight feature, the first weight feature and the second weight feature are fused to obtain the weight feature, and the sigmoid is used as the activation function to activate the weight feature so as to map the weight value to between 0 and 1 to obtain the final weight connection. Then, the weight values in the weight vector are multiplied by the corresponding channels in the input image to obtain the corresponding attention feature image. Among them, the first weight feature, the second weight feature, and the weight vector have the same size.

[0091] In one embodiment, two attention allocation models can be set, which are respectively connected to the detail feature extraction model and the semantic feature extraction model to respectively allocate attention weights to each channel of the detail feature image and the semantic feature image. During training, the two attention allocation models are respectively trained so that the fully connected layers in the two attention allocation models have different weights, so that the extracted weights are more matched with the corresponding input images.

[0092] The image fusion unit 164 is used to fuse the detail feature image, the detail attention feature image, the semantic feature image, and the semantic attention feature image multiple times to obtain the quality feature image;

[0093] Specifically, as Figure 8 shown, the image fusion unit 164 includes:

[0094] A first fusion module 1641, configured to fuse the detail feature image and the detail attention feature image to obtain a fused detail feature image;

[0095] Specifically, before and after assigning weights to the detail feature image, the size and number of channels of the image do not change. Therefore, the detail feature image and the detail attention feature image have the same size and number of channels and can be fused. In the fused detail feature image obtained after fusion, the pixel value of each pixel point is the sum of the pixel values of the corresponding pixel points in the detail feature image and the detail attention feature image.

[0096] A second fusion module 1642, configured to fuse the semantic feature image and the semantic attention feature image to obtain a fused semantic feature image;

[0097] Specifically, similarly, due to the order of weight assignment, the size and number of channels of the image do not change. Therefore, the semantic feature image and the semantic attention feature image also have the same size and number of channels. In the fused semantic feature image obtained after fusion, the pixel value of each pixel point is the sum of the pixel values of the corresponding pixel points in the fused semantic feature image and the semantic attention feature image.

[0098] A third fusion module 1643, configured to fuse the fused detail feature image and the fused semantic feature image to obtain a quality feature image.

[0099] Specifically, in order to be able to fuse the fused detail feature image and the fused semantic feature image, the fused detail feature image and the fused semantic feature image should also have the same size and number of channels. The size of the detail feature image and the semantic feature image can be adjusted by adjusting the size, number, and convolution stride of the convolution kernels in each convolution layer in the detail feature extraction model and the semantic feature extraction model, so that the detail feature image and the semantic feature image have the same size and number of channels, thereby enabling the fused detail feature image and the fused semantic feature image to have the same size and number of channels. In the quality feature image, the pixel value of each pixel point is the sum of the pixel values of the corresponding pixel points in the fused detail feature image and the fused semantic feature image.

[0100] The picture quality score determination unit 165 is configured to obtain the quality score corresponding to the product picture according to the quality feature image.

[0101] Specifically, after obtaining the quality feature image, the quality feature image is input into a number of interconnected fully connected layers, and appropriate activation functions are used to activate the outputs of each layer, so as to obtain the quality score corresponding to the input product image. The server 160 respectively evaluates the quality scores of the product images collected by each camera 140, and obtains the quality scores corresponding to each product feature image.

[0102] In addition, before using the above models to determine the quality score, historical data should be used correspondingly to divide the training set and the validation set, and the gradient descent method is used to train each model. During training, cross-entropy, MSE, etc. can be used as the error function, but the present application is not limited thereto.

[0103] The product quality recognition device provided by the present application is provided with a shooting table, a bracket, a universal magic hand, multiple cameras, a switch and a server; wherein, the shooting table is used to set the product to be recognized, the bracket is arranged on the shooting table, and multiple cameras are respectively arranged on the shooting table or the bracket through the universal magic hand and shoot from multiple different angles; the switch is connected to multiple cameras and the server through network cables respectively, and is used to transmit multiple product pictures taken by the multiple cameras to the server; the server is used to respectively extract the detail feature image and the semantic feature image of each product picture, and further perform attention distribution and multiple fusion operations to obtain the product quality feature image, and determine the quality score corresponding to each product picture according to the product quality feature image, realizing the determination of the final quality score of the product to be recognized. During the process of determining the quality score, the influence of factors such as light and shooting angle on picture recognition is eliminated, and by setting the switch and making the recognition algorithm in the form of an executable file, the offline recognition of product quality is realized.

[0104] Based on the above embodiments, further, as Figure 9 shown, in the product quality recognition device provided by the present application, the server 160 further includes:

[0105] A final quality score determination unit 166, configured to determine the final quality score of the product to be recognized according to the quality scores corresponding to each product picture and a preset quality score fitting model.

[0106] Specifically, after obtaining the quality scores corresponding to each product picture, multiple quality scores can be further processed to obtain a uniquely determined final quality score. The final quality score can be calculated by the following formula:

[0107] y = a1x1 + a2x2 + … + a n x n (1)

[0108] where y is the final quality score, x1, x2, …, x nare the quality scores corresponding to each product picture, a1, a2, …, a n are constant coefficients and can be obtained by fitting historical data.

[0109] The product quality identification device provided by this application determines a unique final quality score through the final quality score determination unit 166 according to the quality scores corresponding to each product picture, which more intuitively and accurately reflects the quality of the product.

[0110] Based on the above embodiments, further, as Figure 10 shown, in the product quality identification device provided by this application, the server 160 further includes:

[0111] A pass rate calculation module 167, which calculates the pass rate of the products according to each identification result and the minimum product quality standard when the identification of a preset number of products is completed.

[0112] Specifically, the server 160 can record the quality scores of each product and determine the pass rate of this batch of products according to the quality scores of each product. Among them, products with quality scores greater than or equal to the minimum product quality standard are regarded as qualified, and the pass rate is the ratio of the number of qualified products to the total number of products in this batch.

[0113] The product quality identification device provided by this application can more intuitively understand the overall quality of a batch of products through the pass rate calculation module 167.

[0114] Based on the above embodiments, further, as Figure 1 shown, the product quality identification device provided by this application further includes: a barcode reader 180, which is connected to the switch 150 through a network cable and is used to read the identification code on the product to be identified.

[0115] Specifically, an identification code label can be pre - pasted on the product to be identified, and the identification code of each product to be identified is different. When the quality of the product to be identified is identified, the barcode reader 180 is used to read the identification code label, so as to ensure that the identified quality score corresponds to the product to be identified one by one. Among them, the identification code label can be, for example, an RFID label, and the barcode reader 180 can be, for example, an RFID card issuer and reader integrated machine, but this application is not limited thereto.

[0116] The product quality identification device provided by this application, by setting the barcode reader 180, while realizing the one - to - one correspondence between the product and the quality score, avoids manually entering the product number, greatly reduces the error rate of number entry, and improves the entry efficiency.

[0117] Based on the above embodiments, further, as Figure 1As shown in the figure, the product quality identification device provided by this application further includes: a light source 190, which illuminates the product to be identified, further excluding the interference of light factors on the identification result.

[0118] In addition, a power supply connected to multiple cameras can be set according to actual needs to meet the power consumption requirements of each camera. Among them, the power supply can be a power cord or a storage battery, etc., and this application does not limit this.

[0119] Based on the same inventive concept, the embodiments of this application also provide a product quality identification method, which is implemented based on the device described in the above embodiments, as described in the following embodiments. Since the principle of the product quality identification method to solve problems is similar to that of the product quality identification device, the implementation of the product quality identification method can refer to the implementation of the above embodiments, and the repeated parts will not be described again.

[0120] Figure 11 is a flowchart of the product quality identification method provided by some embodiments of this application, as Figure 11 As shown in the figure, the product quality identification method provided by this application includes:

[0121] S1101: Obtain the number of the product to be identified and input it into the server;

[0122] S1102: Turn on multiple cameras and obtain multiple pictures of the product to be identified from different angles;

[0123] S1103: The switch transmits the multiple pictures collected by the cameras to the server;

[0124] S1104: The server stores the multiple pictures into the document corresponding to the number, and respectively extracts the detailed feature images and semantic feature images of each product picture;

[0125] S1105: Perform attention allocation and multiple fusion operations on the detailed feature images and semantic feature images to obtain the product quality feature images, and determine the quality scores corresponding to each product picture according to the product quality feature images;

[0126] S1106: Determine the final quality score of the product to be identified according to the product quality corresponding to each product picture.

[0127] The product quality identification method provided by this application obtains the number of the product to be identified and inputs it into the server, turns on multiple cameras and obtains multiple pictures of the product to be identified from different angles. The switch transmits the multiple pictures collected by the cameras to the server, and the server stores the multiple pictures in the document corresponding to the number. The detailed feature images and semantic feature images of each product picture are respectively extracted, and attention allocation and multiple fusion operations are performed on the detailed feature images and semantic feature images to obtain the product quality feature image. The quality score corresponding to each product picture is determined according to the product quality feature image, and the final quality score of the product to be identified is determined according to the product quality corresponding to each product picture, realizing the determination of the final quality score of the product to be identified. During the process of determining the quality score, the influence of factors such as light and shooting angle on picture recognition is eliminated, and by setting a switch and making the recognition algorithm in the form of an executable file, the offline identification of product quality is realized.

[0128] The following is a detailed explanation of each step:

[0129] S1101: Obtain the number of the product to be identified and input it into the server;

[0130] Specifically, different numbers can be set for each product to be identified. Before taking pictures of the product to be identified, the number of the product to be identified is input into the server. After the server receives the pictures taken by multiple cameras from different angles, the pictures are stored in the file or folder corresponding to the number.

[0131] In an embodiment, the number of the picture can be pasted on the product to be identified in the form of an identification code label. At this time, as Figure 12 shown, S1101 includes:

[0132] S1201: The barcode reader identifies the identification code on the product to be identified to obtain the corresponding code of the product to be identified;

[0133] Specifically, the identification code label can be, for example, an RFID label, and the barcode reader 180 can be, for example, an RFID card issuer and reader integrated machine, but this application is not limited thereto.

[0134] S1202: The switch transmits the code identified by the barcode reader to the server.

[0135] Specifically, after the barcode reader identifies the number corresponding to the product to be identified, it transmits the number to the switch, and the switch transmits it to the server. After the server receives the number of the product to be identified, it creates a corresponding file or folder to receive the subsequent product pictures.

[0136] S1102: Turn on multiple cameras and obtain multiple pictures of the product to be identified from different angles;

[0137] Specifically, multiple cameras can take pictures from multiple specific angles, thus preventing the difference in shooting angles from affecting the product recognition results during each shooting. In addition, multiple specific angles can be set so that at least two cameras correspond to different shooting angles, thereby obtaining product pictures from multiple different angles and preventing inaccurate product quality recognition due to the occlusion of some features of the product when the product quality is recognized from a single angle.

[0138] In one embodiment, multiple cameras can take pictures from a horizontal angle (0° frontal view parallel to the ground), a vertical downward view angle (180° downward view perpendicular to the ground), and an inclined angle (for example, shooting obliquely at 45° to the ground), but the present application is not limited thereto.

[0139] S1103: The switch transmits multiple pictures collected by the cameras to the server;

[0140] Specifically, the interfaces of the server are often limited. By setting the switch to be connected to each camera through multiple interfaces respectively, and then transmitting the pictures input by each camera to the server centrally through a single interface, the problem of insufficient server interfaces is solved. In addition, the switch is connected to each camera and the server through network cables for wired communication, and information transmission can also be realized in the offline state.

[0141] S1104: The server stores multiple pictures in the corresponding numbered documents, and respectively extracts the detailed feature images and semantic feature images of each product picture;

[0142] Specifically, after receiving multiple pictures input by the switch, the server transmits the pictures to the documents established in advance according to the numbers. And the pictures in the documents are respectively input into the pre-established detailed feature extraction model and semantic feature extraction model for feature extraction to obtain the corresponding detailed feature images and semantic feature images.

[0143] S1105: Perform attention allocation and multiple fusion operations on the detailed feature images and semantic feature images to obtain product quality feature images, and determine the quality scores corresponding to each product picture according to the product quality feature images;

[0144] Specifically, the server respectively performs attention allocation on the detailed feature images and semantic feature images of each product picture, and fuses the detailed feature images after attention allocation with the detailed feature images before allocation to obtain fused detailed feature images, fuses the semantic feature images after attention allocation with the semantic feature images before allocation to obtain fused semantic feature images, and performs further fusion operations on the fused detailed feature images and fused semantic feature images to obtain product quality feature images. Finally, the corresponding quality scores are determined respectively according to the quality feature images of each product picture.

[0145] S1106: Determine the final quality score of the product to be recognized according to the product quality corresponding to each product picture.

[0146] Specifically, after obtaining the quality scores corresponding to each product picture, multiple quality scores can be further processed to obtain a uniquely determined final quality score. The final quality score can be calculated through formula (1).

[0147] The product quality recognition method provided by this application includes: obtaining the number of the product to be recognized and entering it into the server, turning on multiple cameras and obtaining multiple pictures of the product to be recognized from different angles, transmitting the multiple pictures collected by the cameras to the server by a switch, and storing the multiple pictures into the document corresponding to the number by the server, respectively extracting the detailed feature images and semantic feature images of each product picture, performing attention allocation and multiple fusion operations on the detailed feature images and semantic feature images to obtain product quality feature images, determining the quality scores corresponding to each product picture according to the product quality feature images, and determining the final quality score of the product to be recognized according to the product quality corresponding to each product picture, thus realizing the determination of the final quality score of the product to be recognized. During the process of determining the quality score, the influence of factors such as light and shooting angle on picture recognition is eliminated, and by setting up a switch and making the recognition algorithm in the form of an executable file, offline recognition of product quality is realized.

[0148] On the basis of the above embodiments, further, the product quality recognition method provided by this application further includes:

[0149] After completing the recognition of a preset number of products, the server calculates the qualified rate of the products according to each recognition result and the lowest product quality standard.

[0150] Specifically, the server can also record the quality scores of each product. After a certain number of products are recognized, the qualified rate of this batch of products is determined according to the quality scores of each product in this batch. Among them, products with quality scores greater than or greater than or equal to the lowest product quality standard are regarded as qualified, and the qualified rate is the ratio of the number of qualified products to the total number of products in this batch.

[0151] The product quality recognition method provided by this application can more intuitively understand the overall quality situation of a batch of products by calculating the qualified rate.

[0152] In the description of this specification, the descriptions with reference to the terms "one embodiment", "a specific embodiment", "some embodiments", "for example", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0153] The specific embodiments described above further elaborate on the object, technical solution, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A product quality identification device, characterized in that, Including: A shooting table, a bracket, a universal magic arm, multiple cameras, a switch, and a server; The bracket is arranged on the shooting table, and the shooting table is used for arranging the product to be recognized; The multiple cameras are respectively arranged on the shooting table or the bracket through the universal magic arm and shoot from multiple different angles; The switch is respectively connected to the multiple cameras and the server through network cables and is used for transmitting multiple product pictures taken by the multiple cameras to the server; The server is used for respectively extracting the detailed feature image and the semantic feature image of each product picture, performing attention allocation and multiple fusion operations on the detailed feature image and the semantic feature image to obtain a product quality feature image, and determining the quality score corresponding to each product picture according to the product quality feature image; Determining the final quality score of the product to be recognized according to the product quality corresponding to each product picture; The server includes: A detailed feature image extraction unit, which is used for inputting the received product picture into a detailed feature extraction model to obtain a detailed feature image; A semantic feature image extraction unit, which is used for inputting the received product picture into a semantic feature extraction model to obtain a semantic feature image; An attention allocation unit, which is used for respectively inputting the detailed feature image and the semantic feature image into an attention allocation model to obtain a detailed attention feature image and a semantic attention feature image; An image fusion unit, which is used for performing multiple fusions on the detailed feature image, the detailed attention feature image, the semantic feature image, and the semantic attention feature image to obtain a quality feature image; A picture quality score determination unit, which is used for obtaining the quality score corresponding to the product picture according to the quality feature image; The image fusion unit includes: A first fusion module, which is used for fusing the detailed feature image and the detailed attention feature image to obtain a fused detailed feature image; A second fusion module, which is used for fusing the semantic feature image and the semantic attention feature image to obtain a fused semantic feature image; A third fusion module, which is used for fusing the fused detailed feature image and the fused semantic feature image to obtain the quality feature image.

2. The product quality identification device according to claim 1, characterized in that, The multiple cameras respectively shoot from a horizontal angle, a vertical top-down angle, and an inclined angle.

3. The product quality identification device according to claim 1, characterized in that, It also includes: A camera control device, which is respectively connected to the multiple cameras and is used for controlling the multiple cameras to shoot.

4. The product quality identification device according to claim 1, characterized in that, The product quality recognition device also includes: a barcode reader, which is connected to the switch through a network cable and is used for reading the identification code on the product to be recognized.

5. The product quality identification device according to claim 1, characterized in that, The server also includes: A final quality score determination unit, which is used for determining the final quality score of the product to be recognized according to the quality score corresponding to each product picture and a preset quality score fitting model.

6. The product quality identification device according to claim 1, characterized in that, The server also includes: a pass rate calculation module, which calculates the pass rate of the products according to each recognition result and the lowest product quality standard when the recognition of a preset number of products is completed.

7. The product quality identification device according to claim 1, characterized in that, It also includes: A power supply, which is connected to the multiple cameras.

8. A product quality identification method, implemented based on the product quality identification device according to any one of claims 1-7, characterized in that, Including: Obtaining the number of the product to be recognized and inputting it into the server; Turning on the multiple cameras and obtaining multiple pictures of the product to be recognized from different angles; The switch transfers the multiple pictures collected by the camera to the server; The server stores the multiple pictures into the document corresponding to the number, and respectively extracts the detailed feature image and semantic feature image of each product picture; Perform attention allocation and multiple fusion operations on the detailed feature image and semantic feature image to obtain a product quality feature image, and determine the quality score corresponding to each product picture according to the product quality feature image; Determine the final quality score of the product to be recognized according to the product quality corresponding to each product picture.

9. The product quality identification method according to claim 8, characterized in that, The obtaining of the number of the product to be recognized and the inputting into the server includes: The barcode reader recognizes the identification code on the product to be recognized to obtain the code corresponding to the product to be recognized; The switch transfers the code recognized by the barcode reader to the server.

10. The product quality identification method according to claim 8, characterized in that, It further includes: After recognizing a preset number of products, the server calculates the pass rate of the products according to each recognition result and the minimum product quality standard.

Images