Warehouse commodity qualification rate detection method and device and readable storage medium

Abstract

The invention provides a warehouse commodity qualification rate detection method and device and a readable storage medium, and the method comprises the steps: obtaining the appearance image data of a to-be-detected commodity, and the environment data of a warehouse where the to-be-detected commodity is located; performing feature extraction on the appearance image data and the environment data to obtain image features and environment features; performing feature fusion on the image features and the environment features to obtain a comprehensive feature vector of the to-be-detected commodity; and inputting the comprehensive feature vector into a pre-trained commodity qualification rate detection model for detection to obtain a qualification rate detection result of the to-be-detected commodity. The problems that a traditional manual sampling inspection mode is low in efficiency, high in subjectivity, difficult to comprehensively cover and the like are effectively solved, so that the overall benefit of warehouse operation is improved, the commodity quality is guaranteed, the cost risk caused by unqualified commodities is reduced, and the competitiveness and sustainable development capacity of enterprises in market competition are enhanced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a warehouse commodity qualification rate detection method, device and readable storage medium. Background Art

[0002] With the rapid development of modern warehousing and logistics industries, the inspection of warehouse commodity qualification rate has become an important link in ensuring commodity quality and customer satisfaction.

[0003] The inspection of product qualification rate helps to timely discover quality problems, ensure that the products entering the market meet national standards and industry requirements, and reduce the risks of consumers during use. High-quality products can meet consumer needs, improve customer satisfaction, and thus enhance the company's brand image and market share. By testing the qualification rate of products, companies can understand the production level of suppliers, strengthen control over the upstream links of the supply chain, and reduce supply chain risks. Timely discovery of unqualified products can avoid returns, exchanges, complaints, and other events caused by quality problems, and reduce corporate operating costs.

[0004] At present, traditional detection methods often rely on manual inspection or simple machine vision technology, but these methods have problems of low efficiency and high error rate. Summary of the invention

[0005] The technical problem to be solved by the present application is to provide a warehouse commodity qualification rate detection method, device and readable storage medium to solve the problems existing in the prior art in view of the above-mentioned deficiencies in the prior art.

[0006] In a first aspect, the present application provides a method for detecting the qualified rate of goods in a warehouse,

[0007] Methods include:

[0008] S1. Obtain appearance image data of the commodity to be inspected, and environmental data of the warehouse where the commodity to be inspected is located;

[0009] S2, extracting features from the appearance image data and the environment data respectively to obtain image features and environment features;

[0010] S3, performing feature fusion on the image features and the environmental features to obtain a comprehensive feature vector of the commodity to be detected;

[0011] S4. Input the comprehensive feature vector into a pre-trained commodity qualification rate detection model for detection to obtain a qualification rate detection result of the commodity to be detected.

[0012] In some embodiments, in S1, the environmental data includes at least one of temperature data, humidity data, light data, air quality data, ventilation data, pest data, vibration data, noise data, and power system data.

[0013] In some embodiments, S2 includes:

[0014] Extracting features from the appearance image data using a Unet network to obtain image features;

[0015] The feature extraction of the environmental data is performed through the Encoder part of the Transformer to obtain the environmental features.

[0016] In some embodiments, S3 includes:

[0017] The image features extracted by the Unet network and the environmental features extracted by the Encoder part of the Transformer are spliced ​​in the feature dimension to obtain a comprehensive feature vector, which includes the appearance image information of the commodity to be detected and the environmental information of the warehouse where the commodity to be detected is located.

[0018] In some embodiments, the product qualification rate detection model is a deep learning model based on a long short-term memory network (LSTM) and a convolutional neural network (CNN).

[0019] In some embodiments, the product qualification rate detection model is trained by the following process:

[0020] Acquire an original data set with marking information, wherein the original data set includes appearance image data of the commodity and environmental data of the warehouse where the commodity is located, and the marking information is whether the commodity is qualified or unqualified;

[0021] Dividing the original data set into a training set, a validation set and a test set;

[0022] An initial model is built based on LSTM and CNN, and the training set is used for model training, the validation set is used for model verification, and the test set is used for model testing to obtain a trained product qualification rate detection model.

[0023] In some embodiments, after S1 and before S2, the process further includes:

[0024] Preprocessing the appearance image data and the environment data;

[0025] Wherein, the preprocessing of the appearance image data includes at least one of image cropping, scaling, and normalization processing;

[0026] The preprocessing of the environmental data includes at least one of removing outliers, removing missing values, and data standardization.

[0027] In a second aspect, the present application provides a warehouse commodity qualification rate detection device, the device comprising:

[0028] A data acquisition module, which is configured to acquire appearance image data of the commodity to be detected, and environmental data of the warehouse where the commodity to be detected is located;

[0029] A feature extraction module, configured to extract features from the appearance image data and the environment data respectively to obtain image features and environment features;

[0030] A feature fusion module, which is configured to fuse the image features and the environmental features to obtain a comprehensive feature vector of the commodity to be detected;

[0031] The commodity detection module is configured to input the comprehensive feature vector into a pre-trained commodity qualification rate detection model for detection, and obtain a qualification rate detection result of the commodity to be detected.

[0032] In a third aspect, the present application provides a warehouse commodity qualification rate detection device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the warehouse commodity qualification rate detection method described in the first aspect above.

[0033] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the warehouse commodity qualification rate detection method described in the first aspect is implemented.

[0034] The present application provides a warehouse commodity qualification rate detection method, device and readable storage medium, the method comprising: obtaining appearance image data of the commodity to be detected, and environmental data of the warehouse where the commodity to be detected is located; performing feature extraction on the appearance image data and the environmental data respectively to obtain image features and environmental features; performing feature fusion on the image features and the environmental features to obtain a comprehensive feature vector of the commodity to be detected; inputting the comprehensive feature vector into a pre-trained commodity qualification rate detection model for detection to obtain a qualified rate detection result of the commodity to be detected. The present application provides a warehouse commodity qualification rate detection method, which combines the multi-dimensional collection and feature extraction of image data and environmental data, and uses an advanced deep learning network architecture for feature fusion and model construction, aiming to provide an efficient, accurate and intelligent commodity quality detection solution for warehouse management, effectively solving the problems of low efficiency, strong subjectivity and difficulty in comprehensive coverage in traditional manual sampling methods, thereby improving the overall efficiency of warehouse operations, ensuring commodity quality, reducing the cost risk caused by unqualified commodities, and enhancing the competitiveness and sustainable development capabilities of enterprises in market competition. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0036] Figure 1 A flowchart of a warehouse commodity qualification rate detection method provided in an embodiment of the present application;

[0037] Figure 2 A flowchart of another warehouse commodity qualification rate detection method provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of the structure of a warehouse commodity qualification rate detection device provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of the structure of another warehouse commodity qualification rate detection device provided in an embodiment of the present application.

[0040] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the technical solution of the present application, the implementation mode of the present application will be further described in detail below with reference to the accompanying drawings.

[0042] It should be understood that the specific embodiments and drawings described herein are only used to explain the present application, rather than to limit the present application.

[0043] It can be understood that, in the absence of conflict, the various embodiments in the present application and the various features in the embodiments can be combined with each other.

[0044] It can be understood that, for the convenience of description, the drawings of the present application only show the parts related to the present application, while the parts unrelated to the present application are not shown in the drawings.

[0045] It can be understood that each unit and module involved in the embodiments of the present application may correspond to only one physical structure, or may be composed of multiple physical structures, or multiple units and modules may be integrated into one physical structure.

[0046] It can be understood that the terms "first", "second", etc. in the embodiments of the present application are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0047] It is understandable that, in the absence of conflict, the functions and steps marked in the flowcharts and block diagrams of the present application may occur in an order different from that marked in the drawings.

[0048] It is understood that the flowcharts and block diagrams of the present application illustrate the possible architectures, functions, and operations of the systems, devices, equipment, and methods according to the various embodiments of the present application. Among them, each box in the flowchart or block diagram may represent a unit, module, program segment, code, which contains executable instructions for implementing the specified functions. Moreover, each box or combination of boxes in the block diagram and flowchart may be implemented by a hardware-based system that implements the specified functions, or by a combination of hardware and computer instructions.

[0049] It can be understood that the units and modules involved in the embodiments of the present application can be implemented by software or hardware, for example, the units and modules can be located in a processor.

[0050] In order to improve detection efficiency and accuracy, multi-feature fusion and deep learning technologies have been widely used in the detection of warehouse commodity qualification rates in recent years.

[0051] 1. Multi-feature fusion technology

[0052] Multi-feature fusion technology refers to the organic combination of image features from different sources or types to form a more comprehensive and robust feature descriptor. This method can significantly improve the accuracy and efficiency of image matching by integrating the advantages of multiple features.

[0053] In the detection of the qualified rate of warehouse goods, multi-feature fusion technology can be applied to the detection of appearance, size, texture, color and other aspects of goods. For example, when detecting the appearance of goods, multiple features such as the edge, texture, shape and so on of the goods can be fused to construct a more discriminating feature descriptor, thus improving the accuracy and robustness of detection.

[0054] 2. Deep Learning Technology

[0055] Deep learning technology is a powerful machine learning technology that can automatically extract effective features from data by building a multi-layer neural network model and use it for prediction or decision-making. In the inspection of the qualified rate of warehouse goods, deep learning technology can achieve high-quality and efficient inspection of goods.

[0056] The deep learning model has a powerful feature learning capability and can automatically extract effective features from product images or data, such as the product's shape, texture, color, etc. At the same time, the deep learning model can also process high-dimensional data, capture nonlinear relationships in the data, and has strong robustness and can effectively resist the influence of noise and outliers.

[0057] In the detection of the qualified rate of warehouse goods, deep learning technology can be applied in many aspects. For example, when detecting the size of goods, deep learning models can be used to accurately measure the image of the goods to ensure that the size of the goods meets the standards. In terms of detecting the performance of goods, deep learning models can be used to quantify the performance indicators of goods, such as durability and stability.

[0058] 3. Combination of Multi-Feature Fusion and Deep Learning

[0059] Combining multi-feature fusion technology with deep learning technology can further improve the inspection effect of warehouse commodity qualification rate. Through multi-feature fusion, more comprehensive and robust feature descriptors can be obtained, providing richer input information for deep learning models. Deep learning models can automatically extract effective features from this information and make predictions or decisions.

[0060] In the detection of the qualified rate of warehouse goods, multiple features of goods can be integrated, such as appearance, size, texture, color, etc. Then, these fused features can be learned and predicted using a deep learning model. By training and optimizing the deep learning model, high-precision and high-efficiency detection of goods can be achieved, and the accuracy of the detection of the qualified rate of goods can be improved.

[0061] However, the above technology still has the following disadvantages:

[0062] 1. Dataset Issues

[0063] Small sample size: In the field of warehouse commodity inspection, especially for specific commodities or defects, the number of samples is often limited. This may cause the deep learning model to overfit during the training process, that is, the model performs well on the training set, but has poor generalization ability in actual inspection.

[0064] Difficulty in data labeling: Accurately labeling product images requires a lot of manpower and time, especially for complex defects or the fusion of multiple features. The accuracy of labeling directly affects the detection effect of the model.

[0065] 2. Feature Fusion Problem

[0066] Improper feature selection: During the feature fusion process, if the selected features are not comprehensive or representative enough, the detection performance of the model may be affected. At the same time, too many features will increase the complexity and computational complexity of the model.

[0067] Feature fusion method: There are many feature fusion methods, such as simple concatenation, weighted summation, etc. However, different fusion methods may have different effects on the performance of the model, and how to choose a suitable fusion method is still a challenge.

[0068] 3. Deep Learning Model Problems

[0069] Model complexity: Deep learning models are usually highly complex and require a lot of computing resources and time for training and reasoning. This may be a bottleneck for warehouse commodity detection scenarios with high real-time requirements.

[0070] Model robustness: In practical applications, product images may be affected by various factors such as lighting, noise, and occlusion, which may lead to a decrease in the detection performance of the model. Therefore, improving the robustness of the model is one of the current research focuses.

[0071] In view of the above shortcomings, this application proposes an innovative method for detecting the qualified rate of goods in a warehouse environment. The main ideas of this application are as follows:

[0072] First, in order to solve the problem of small sample size, data enhancement techniques are used, such as rotating, flipping, cropping, adding noise, etc. to existing images to expand the number of samples. At the same time, transfer learning is used to initialize the network model in this application with the help of model parameters pre-trained on large-scale general image datasets to alleviate the overfitting phenomenon caused by limited samples. For the difficulty of data annotation, a semi-automatic annotation tool is developed, which first uses image recognition algorithms to preliminarily identify product features and possible defective areas, and then manually reviews and corrects them to improve annotation efficiency and accuracy.

[0073] In terms of feature fusion, the feature importance evaluation algorithm is used to screen out features that have a significant impact on product qualification rate detection, avoiding the impact of improper feature selection on model performance. At the same time, in terms of feature fusion, an adaptive weighted fusion strategy is adopted to dynamically adjust the weights according to the importance of different features in different environments and product categories to improve the fusion effect.

[0074] For deep learning model problems, we adopt lightweight network design ideas when building models, optimize LSTM and CNN structures, reduce unnecessary parameters and computing layers, improve model training and reasoning speed, and meet the real-time requirements of warehouse commodity detection. In addition, we add adversarial training mechanisms to the model training process to generate adversarial samples, so that the model can better cope with interference factors such as lighting, noise, and occlusion, and enhance the robustness of the model, thereby achieving efficient, accurate, and stable detection of commodity qualification rates in warehouse environments, effectively improving the intelligence level of warehouse management and quality control capabilities.

[0075] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0076] The present application provides a method for detecting the qualified rate of goods in a warehouse. The working process of the method can be implemented by electronic devices, such as computers, handheld smart terminals, etc. For the convenience of explanation, the embodiments of the present application are described with the method execution subject being a computer.

[0077] Figure 1 A schematic diagram of a warehouse commodity qualification rate detection method provided in an embodiment of the present application, Figure 2 Another schematic diagram of the warehouse commodity qualification rate detection method provided in the embodiment of the present application is as follows Figure 1 as well as Figure 2 As shown, the present application provides a method for detecting the qualified rate of warehouse goods, and the method includes S1-S4, which are as follows:

[0078] S1. Obtain appearance image data of the commodity to be inspected, and environmental data of the warehouse where the commodity to be inspected is located;

[0079] Among them, high-definition cameras can be used to shoot goods in the warehouse from multiple angles to obtain appearance image data of the goods. Image acquisition is carried out at different shelf positions and lighting conditions to ensure data diversity.

[0080] In some embodiments, in S1, the environmental data includes at least one of temperature data, humidity data, light data, air quality data, ventilation data, pest data, vibration data, noise data, and power system data.

[0081] Specifically, the environmental characteristic data of commodity warehouses mainly include the following types:

[0082] (1) Temperature data: The temperature in the warehouse is crucial for certain commodities (such as food, medicine, chemicals, etc.) and needs to be monitored in real time and controlled within an appropriate range.

[0083] (2) Humidity data: Humidity is also important for the storage of goods. Too high or too low humidity may cause damage to the goods.

[0084] (3) Lighting data: Some products may need to be protected from direct sunlight, so it is also important to monitor the light intensity in the warehouse.

[0085] (4) Air quality data: including the concentration of harmful gases (such as carbon dioxide, hydrogen sulfide, etc.) and dust content, to ensure that the air quality in the warehouse meets the standards.

[0086] (5) Ventilation: Good ventilation can ensure the stability of the warehouse environment and is also necessary for the preservation of goods.

[0087] (6) Pest monitoring: For commodities such as food and textiles, it is necessary to monitor whether there is any pest intrusion.

[0088] (7) Vibration data: Certain fragile goods require monitoring of vibration conditions within the warehouse to avoid damage caused by vibration.

[0089] (8) Noise data: The noise level inside the warehouse also needs to be controlled within a certain range to ensure employee health and work efficiency.

[0090] (9) Power system data: including voltage, current, energy consumption, etc., to ensure stable and efficient power supply.

[0091] The impact of the above-mentioned environmental data on the product status is as follows:

[0092] (1) Temperature data:

[0093] Impact: Too high a temperature may cause certain goods (such as food, medicine, and cosmetics) to deteriorate, while too low a temperature may cause certain liquid goods to freeze and plastic or rubber products to become brittle.

[0094] Specific manifestations: For example, medicines may lose their activity at high temperatures, and chocolate may melt and deform at high temperatures.

[0095] (2) Humidity data:

[0096] Impact: Too high humidity may cause the product to mold, rust, or cause the circuit board to short-circuit due to moisture, while too low humidity may cause the product to dry out and crack.

[0097] Specific manifestations: For example, paper tends to become soft and moldy in a high humidity environment, and wooden products may swell and deform.

[0098] (3) Lighting data:

[0099] Impact: Prolonged exposure to strong light may cause some products to fade or age.

[0100] How it can happen: For example, artwork and prints may fade in direct sunlight, and plastics may become brittle.

[0101] (4) Air quality data:

[0102] Impact: Harmful gases and dust may contaminate goods, especially precision electronic equipment and food.

[0103] Specific manifestations: For example, hydrogen sulfide may corrode metals, and dust may clog the heat dissipation holes of electronic equipment.

[0104] (5) Ventilation conditions:

[0105] Impact: Poor ventilation may lead to excessive humidity, uneven temperature and accumulation of harmful gases in the warehouse.

[0106] Specific manifestations: For example, poor ventilation may cause food to produce odors and electronic products to become damp.

[0107] (6) Pest monitoring:

[0108] Impact: Pests may eat or contaminate goods such as food and textiles.

[0109] How it manifests: For example, rats and cockroaches can damage packaging and contaminate stored food.

[0110] (7) Vibration data:

[0111] Impact: Frequent or strong vibrations may damage fragile goods or precision equipment.

[0112] Specific manifestations: For example, vibrations may cause glass products to break and precision instruments to become inaccurate.

[0113] (8) Noise data:

[0114] Impact: Long-term exposure to high noise environments may affect the physical and mental health of warehouse workers, and indirectly affect the management and handling of goods.

[0115] Specific manifestations: For example, high noise levels may cause worker fatigue and increase the risk of operational errors.

[0116] (9) Power system data:

[0117] Impact: Unstable power supply may cause refrigeration equipment to fail, insufficient lighting may affect work efficiency, and electronic monitoring equipment may fail.

[0118] How it manifests: For example, power outages could cause refrigerated foods to spoil.

[0119] The collection of the above data usually relies on various sensors and monitoring equipment, and the timestamps are matched with the product images taken at the corresponding time to ensure the relevance of environmental data and product image data. Real-time monitoring and data recording are achieved through IoT technology, so that managers can understand the warehouse environment status in a timely manner and take corresponding measures. For example, multiple temperature and humidity sensors are distributed in the warehouse to collect environmental temperature and humidity data in real time.

[0120] S2, extracting features from the appearance image data and the environment data respectively to obtain image features and environment features;

[0121] In some embodiments, S2 includes:

[0122] Extracting features from the appearance image data using a Unet network to obtain image features;

[0123] The feature extraction of the environmental data is performed through the Encoder part of the Transformer to obtain the environmental features.

[0124] Specifically, when extracting image features, the Unet network is used to extract features from the collected product images. The encoder part of the Unet network downsamples the image and extracts image features at different levels. The decoder part upsamples and fuses these features to finally obtain a vector appearance feature representation of the image. This structure can effectively capture the details and overall structural information in the product image, and plays an important role in detecting product appearance defects such as scratches, deformations, stains, etc.

[0125] Among them, the Unet network is a widely used convolutional neural network architecture, known for its unique "U"-shaped structure. The network effectively combines local details and global contextual information of the image through downsampling and upsampling of features, achieving high-precision pixel-level classification. The Unet network is characterized by a symmetrical contraction path (encoder) and expansion path (decoder). The contraction path is used to extract image features, and the expansion path is used to accurately locate the segmentation boundary. In addition, Unet splices the feature maps of the encoder and decoder through jump connections, thereby retaining rich edge information, allowing the network to perform well in segmenting complex structures.

[0126] In addition, when extracting environmental features, the Encoder part of the Transformer is used to extract features from environmental data. Various types of environmental data are organized into input sequences according to time series and input into the Encoder of the Transformer. The Encoder of the Transformer can learn the complex relationships and potential patterns in environmental data through a multi-head self-attention mechanism, and extract vector representations that reflect environmental features, such as the changing trends of various environmental parameters, abnormal fluctuations, and other information.

[0127] Among them, the Transformer model, as a revolutionary innovation in the field of natural language processing, has completely changed the way of sequence modeling with its unique self-attention mechanism and encoder-decoder architecture. The Transformer model uses a multi-head attention mechanism to achieve a deep understanding of the complex relationship between each element in the sequence, while using position encoding to ensure the effective transmission of word order information. This model not only far exceeds traditional recurrent neural networks in parallel computing capabilities, but also shows excellent performance in handling long-distance dependency problems, bringing unprecedented breakthroughs to tasks such as machine translation and text generation.

[0128] S3, performing feature fusion on the image features and the environmental features to obtain a comprehensive feature vector of the commodity to be detected;

[0129] In some embodiments, S3 includes:

[0130] The image features extracted by the Unet network and the environmental features extracted by the Encoder part of the Transformer are spliced ​​in the feature dimension to obtain a comprehensive feature vector, which includes the appearance image information of the commodity to be detected and the environmental information of the warehouse where the commodity to be detected is located.

[0131] Specifically, the image vector appearance features extracted by Unet and the environmental features extracted by Transformer Encoder are concatenated in the feature dimension to form a comprehensive feature vector. The concatenated feature vector contains both the appearance information of the product and the information of its environment, providing a more comprehensive basis for accurately judging the qualified rate of the product.

[0132] S4. Input the comprehensive feature vector into a pre-trained commodity qualification rate detection model for detection to obtain a qualification rate detection result of the commodity to be detected.

[0133] In some embodiments, the product qualification rate detection model is a deep learning model based on a long short-term memory network LSTM (Long Short-Term Memory) and a convolutional neural network CNN (Convolutional Neural Network).

[0134] Specifically, the LSTM layer can process time series data and learn the changing rules of feature vectors in the time dimension. It has advantages in processing commodity inspection data with certain time correlation (such as inspection data of different batches of commodities at different time points). The CNN convolution layer further extracts and abstracts the features in the spatial dimension, enhancing the model's ability to express features. The last layer of the model uses the Softmax classification layer to output the probability of qualified and unqualified commodities.

[0135] In some embodiments, the product qualification rate detection model is trained by the following process:

[0136] Acquire an original data set with marking information, wherein the original data set includes appearance image data of the commodity and environmental data of the warehouse where the commodity is located, and the marking information is whether the commodity is qualified or unqualified;

[0137] Dividing the original data set into a training set, a validation set and a test set;

[0138] An initial model is built based on LSTM and CNN, and the training set is used for model training, the validation set is used for model verification, and the test set is used for model testing to obtain a trained product qualification rate detection model.

[0139] Specifically, the training and application of the product qualification rate detection model in this application mainly includes the following steps:

[0140] (1) Data division: The collected images and environmental data with annotations (qualified or unqualified products) are divided into training set, validation set and test set according to a certain ratio. For example, 70% is used for training, 15% is used for validation, and 15% is used for testing.

[0141] (2) Model training: Use the training set data to train the constructed LSTM+CNN model. During the training process, a suitable loss function (such as the cross entropy loss function) and optimization algorithm (such as the Adam optimization algorithm) are used, and the model parameters are continuously adjusted through the back propagation algorithm, so that the loss of the model on the training set gradually decreases and the accuracy gradually increases.

[0142] (3) Model verification and optimization: During the training process, the model is verified regularly using the validation set data to monitor the model's performance indicators on the validation set (such as accuracy, recall, F1 value, etc.). If the model overfits on the validation set (such as the validation set loss starts to rise while the training set loss continues to fall), the corresponding regularization techniques (such as L1 / L2 regularization, Dropout, etc.) are used to optimize the model to prevent overfitting and improve the generalization ability of the model. At the same time, the model's hyperparameters (such as learning rate, number of hidden layer nodes, etc.) are adjusted based on the feedback from the validation set to obtain the best model performance.

[0143] (4) Model testing and evaluation: Use the test set data to conduct a final test on the trained and optimized model to evaluate the performance of the model in actual application scenarios. The main evaluation indicators include accuracy, recall, F1 value, and confusion matrix, which comprehensively measure the accuracy and reliability of the model in detecting the qualified rate of goods.

[0144] (5) Online testing: Deploy the trained model to the warehouse’s commodity testing system to perform real-time pass rate testing on newly-stocked or regularly sampled commodities. When new commodity images and environmental data are input, the model can quickly output a prediction of whether the commodity is qualified or not, and feed the result back to the warehouse management system so that appropriate measures can be taken in a timely manner (such as isolating, returning, or repairing unqualified commodities).

[0145] (6) Model update: As time goes by and the warehouse environment changes, the type of goods, appearance characteristics, and the impact of the environment on the quality of goods may change. In order to ensure the accuracy and effectiveness of the model, new product images and environmental data are regularly collected and added to the training data set. The model is retrained and optimized so that the model can adapt to new situations and continuously improve its generalization ability.

[0146] In some embodiments, after S1 and before S2, the process further includes:

[0147] Preprocessing the appearance image data and the environment data;

[0148] Wherein, the preprocessing of the appearance image data includes at least one of image cropping, scaling, and normalization processing;

[0149] The preprocessing of the environmental data includes at least one of removing outliers, removing missing values, and data standardization.

[0150] Specifically, data preprocessing is a key step before data application, which can improve the quality of data and thus improve the accuracy of data processing results.

[0151] Among them, the preprocessing of appearance image data mainly includes:

[0152] (1) Image cropping:

[0153] Purpose: Remove unnecessary parts of the image so that the model can focus on key areas.

[0154] Method: Depending on the requirements, you can crop the center of the image, or locate the target object through the target detection algorithm and crop it.

[0155] (2) Image scaling:

[0156] Purpose: Unify the size of the image to facilitate model processing.

[0157] Method: The image can be adjusted to the size required by the model input layer by scaling it proportionally or non-proportionally.

[0158] (3) Normalization processing:

[0159] Purpose: Scale the image pixel values ​​to a fixed range, usually [0, 1] or [-1, 1], to help convergence during model training.

[0160] Method: Divide each pixel value of the image by 255 (assuming the original pixel values ​​are in the range 0-255).

[0161] In addition, the preprocessing of environmental data mainly includes:

[0162] (1) Removing outliers:

[0163] Purpose: Exclude erroneous records or extreme values ​​in the data, which may have a negative impact on model training.

[0164] Methods: Statistical methods such as Z-score normalization can be used to identify outliers and remove them from the dataset.

[0165] (2) Remove missing values:

[0166] Purpose: To ensure the integrity and availability of datasets.

[0167] Method: You can delete records with missing values, or use interpolation, mean filling and other methods to fill in missing values.

[0168] (3) Data standardization processing:

[0169] Purpose: Convert the data into a form with zero mean and unit variance so that the dimensions of different features are consistent and easy to process by the model.

[0170] Methods: Commonly used normalization methods include min-max normalization (scaling the data to the interval [0,1]) and Z-score normalization (converting the data to a form with zero mean and unit variance).

[0171] Through the above preprocessing steps, the quality of data can be effectively improved, providing a good data foundation for subsequent data application processing.

[0172] To facilitate understanding of the present solution, the technical solution of the present application is described in detail below in conjunction with specific embodiments.

[0173] (I) Preparation of Implementation Examples

[0174] Hardware equipment preparation: Install several high-definition cameras in the warehouse to ensure coverage of each shelf area; deploy a temperature and humidity sensor network to ensure the accuracy and real-time nature of data collection; equip high-performance servers for data storage, model training, and online detection calculations.

[0175] Software environment construction: Build a development environment based on a deep learning framework (such as TensorFlow or PyTorch), and install related libraries and tools to build and train models such as Unet, Transformer, LSTM, and CNN.

[0176] (II) Data collection and preprocessing

[0177] According to the above data collection method, the image data and environmental data of the goods in the warehouse were continuously collected over a period of 6 months, and a total of 83,756 images and corresponding environmental data records were collected.

[0178] The collected image data is preprocessed, including image cropping, scaling, and normalization, to meet the input requirements of the Unet network. The environmental data is cleaned to remove outliers and missing values, and standardized to be within an appropriate numerical range.

[0179] (III) Model training and optimization

[0180] According to the above model construction method, the LSTM+CNN model was built using the TensorFlow framework. The image features extracted by Unet and the environmental features extracted by Transformer Encoder were concatenated as the model input. The number of hidden units in the LSTM layer was set to [specific value], the convolution kernel size, step size and other parameters of the CNN convolution layer, and the number of output categories of the Softmax classification layer (qualified and unqualified) were set.

[0181] Use the training set data to train the model, set the number of training rounds to [specific number of rounds], the initial learning rate to [specific value], and use the Adam optimization algorithm and the cross entropy loss function. During the training process, after each training round of [specific number of iterations], use the validation set data to verify the model and record the performance indicators of the model on the validation set. If the model shows signs of overfitting, use L2 regularization (regularization coefficient is [specific value]) and Dropout (dropout rate is [specific value]) technology to optimize the model and adjust the learning rate appropriately.

[0182] After multiple rounds of training and optimization, the model with the best performance on the test set was obtained, with an accuracy of [specific accuracy value], a recall of [specific recall value], and an F1 value of [specific F1 value].

[0183] 4. Model application and effect evaluation

[0184] The trained model is deployed to the warehouse commodity inspection system to test the qualified rate of a batch of newly-arrived commodities. During the inspection process, the system collects commodity images and environmental data in real time and inputs them into the model for prediction. After actual inspection, the model can quickly and accurately determine the qualified rate of commodities, effectively reducing the workload of manual sampling and improving the efficiency and accuracy of warehouse management.

[0185] The effect of the model after a period of application is continuously evaluated, and the model's false positive rate, missed positive rate and other indicators in actual detection are statistically analyzed. Based on the evaluation results, new data is regularly collected to update and optimize the model to ensure that the model always maintains good performance and adapts to the dynamic changes of warehouse environment and commodities.

[0186] The present application provides a method for detecting the qualified rate of goods in a warehouse, which combines the multi-dimensional collection and feature extraction of image data and environmental data, and uses an advanced deep learning network architecture for feature fusion and model construction, aiming to provide warehouse management with an efficient, accurate and intelligent product quality detection solution, and effectively solve the problems of low efficiency, strong subjectivity and difficulty in comprehensive coverage in traditional manual sampling methods, thereby improving the overall efficiency of warehouse operations, ensuring product quality, reducing cost risks caused by unqualified products, and enhancing the competitiveness and sustainable development capabilities of enterprises in market competition.

[0187] It should be understood that, although the various steps in the flowcharts in the above-described embodiments are sequentially displayed according to the indications of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and they can be executed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily to be carried out sequentially, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0188] Figure 3 A schematic diagram of a warehouse commodity qualification rate detection device provided in an embodiment of the present application, such as Figure 3 As shown, the present application provides a warehouse commodity qualification rate detection device, the device comprising:

[0189] A data acquisition module 11, which is configured to acquire appearance image data of the commodity to be detected, and environmental data of the warehouse where the commodity to be detected is located;

[0190] A feature extraction module 12, which is configured to extract features from the appearance image data and the environment data respectively to obtain image features and environment features;

[0191] A feature fusion module 13, which is configured to perform feature fusion on the image features and the environmental features to obtain a comprehensive feature vector of the commodity to be detected;

[0192] The commodity detection module 14 is configured to input the comprehensive feature vector into a pre-trained commodity qualification rate detection model for detection, and obtain a qualification rate detection result of the commodity to be detected.

[0193] Regarding the limitation of the warehouse commodity qualification rate detection device, reference may be made to the limitation of the warehouse commodity qualification rate detection method in the above-mentioned embodiments of the present application, which will not be elaborated in this embodiment.

[0194] Figure 4 Another schematic diagram of the warehouse commodity qualification rate detection device provided in the embodiment of the present application is as follows Figure 4 As shown, in some embodiments, the present application provides a warehouse commodity qualification rate detection device, including a memory 22 and a processor 21, the memory stores a computer program, and the processor is configured to run the computer program to execute the warehouse commodity qualification rate detection method in the above-mentioned embodiments of the present application.

[0195] The memory is connected to the processor, the memory may be a flash memory or a read-only memory or other memory, and the processor may be a central processing unit or a single-chip microcomputer.

[0196] In some embodiments, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the warehouse commodity qualification rate detection method in the above-mentioned embodiments of the present application is implemented.

[0197] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0198] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present application, but the present application is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of the present application, and these modifications and improvements are also considered to be within the scope of protection of the present application.

Claims

1. A method for detecting the qualified rate of warehouse goods, characterized in that: The method comprises: S1. Obtain appearance image data of the commodity to be inspected, and environmental data of the warehouse where the commodity to be inspected is located; S2, extracting features from the appearance image data and the environment data respectively to obtain image features and environment features; S3, performing feature fusion on the image features and the environmental features to obtain a comprehensive feature vector of the commodity to be detected; S4. Input the comprehensive feature vector into a pre-trained commodity qualification rate detection model for detection to obtain a qualification rate detection result of the commodity to be detected.

2. The warehouse commodity qualification rate detection method according to claim 1 is characterized in that: In S1, the environmental data includes at least one of temperature data, humidity data, light data, air quality data, ventilation data, pest data, vibration data, noise data, and power system data.

3. The warehouse commodity qualification rate detection method according to claim 1 is characterized in that: S2, including: Extracting features from the appearance image data using a Unet network to obtain image features; The feature extraction of the environmental data is performed through the Encoder part of the Transformer to obtain the environmental features.

4. The warehouse commodity qualification rate detection method according to claim 3 is characterized in that: S3, including: The image features extracted by the Unet network and the environmental features extracted by the Encoder part of the Transformer are spliced ​​in the feature dimension to obtain a comprehensive feature vector, which includes the appearance image information of the commodity to be detected and the environmental information of the warehouse where the commodity to be detected is located.

5. The warehouse commodity qualification rate detection method according to claim 1 is characterized in that: The product qualification rate detection model is a deep learning model based on the long short-term memory network LSTM and the convolutional neural network CNN.

6. The warehouse commodity qualification rate detection method according to claim 5 is characterized in that: The product qualification rate detection model is trained through the following process: Acquire an original data set with marking information, wherein the original data set includes appearance image data of the commodity and environmental data of the warehouse where the commodity is located, and the marking information is whether the commodity is qualified or unqualified; Dividing the original data set into a training set, a validation set and a test set; An initial model is built based on LSTM and CNN, and the training set is used for model training, the validation set is used for model verification, and the test set is used for model testing to obtain a trained product qualification rate detection model.

7. The warehouse commodity qualification rate detection method according to any one of claims 1 to 6, characterized in that: After S1 and before S2, it also includes: Preprocessing the appearance image data and the environment data; Wherein, the preprocessing of the appearance image data includes at least one of image cropping, scaling, and normalization processing; The preprocessing of the environmental data includes at least one of removing outliers, removing missing values, and data standardization.

8. A warehouse commodity qualification rate detection device, characterized in that: The device comprises: A data acquisition module, which is configured to acquire appearance image data of the commodity to be detected, and environmental data of the warehouse where the commodity to be detected is located; A feature extraction module, configured to extract features from the appearance image data and the environment data respectively to obtain image features and environment features; A feature fusion module, which is configured to fuse the image features and the environmental features to obtain a comprehensive feature vector of the commodity to be detected; The commodity detection module is configured to input the comprehensive feature vector into a pre-trained commodity qualification rate detection model for detection, and obtain a qualification rate detection result of the commodity to be detected.

9. A warehouse commodity qualification rate detection device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the warehouse commodity qualification rate detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the warehouse commodity qualification rate detection method according to any one of claims 1 to 7 is implemented.

Images