Legacy identification method and device, equipment and storage medium

Abstract

The invention relates to the technical field of image recognition, in particular to a legacy recognition method, device and equipment and a storage medium, and the method comprises the steps: building a to-be-trained model, and setting the model parameters of the to-be-trained model; obtaining a plurality of historical site pictures, and respectively carrying out labeling processing and augmentation processing to obtain historical augmentation samples; performing division processing on the historical augmented sample, and inputting the divided historical augmented sample into a to-be-trained model for training to obtain an express identification model; acquiring a real-time to-be-recognized image, and preprocessing the real-time to-be-recognized image to obtain a real-time preprocessed image; inputting the real-time preprocessed image into an express item identification model to obtain an identification result, and judging whether an express item omission problem exists or not according to the identification result; according to the method disclosed by the invention, automatic identification of the express item omission problem can be realized, the complexity and errors of manual inspection are reduced, the accuracy and timeliness of express item processing are ensured, and the management efficiency and management quality of a website are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a legacy part recognition method, device, equipment and storage medium. Background Art

[0002] With the rapid development of e-commerce, the express delivery industry, as its key supporting service, has led to a continuous increase in consumer demand for express delivery services. During special periods such as holidays and promotional seasons, the express delivery business volume has increased sharply. Under this prosperous scene, the express delivery industry has encountered a series of challenges. Specifically, due to the huge workload, it is prone to high error rates, mismatch between efficiency and demand, and low efficiency. These problems are particularly evident under poor management and have become an obstacle to the further development of the express delivery industry.

[0003] During business peak periods, especially during e-commerce promotional events such as Double Eleven and Double Twelve, express delivery outlets face tremendous sorting and delivery pressure; the surge in order volume forces couriers to handle a number of packages far beyond their normal working capacity, which not only significantly reduces operational efficiency, but also the traditional site management method mainly relies on manual inspection and cleaning, which is inefficient and easily affected by human factors, resulting in the omission of scattered express parcels, which in turn affects the quality of express services and customer satisfaction.

[0004] It can be seen that the existing technology still needs to be improved and enhanced. Summary of the invention

[0005] In order to overcome the deficiencies of the prior art, the purpose of the present invention is to provide a method for identifying leftover items, which can realize automatic identification of missing express items, reduce the tediousness and errors of manual inspection, and ensure the accuracy and timeliness of express item processing.

[0006] The first aspect of the present invention provides a method for identifying leftover items, including: building a model to be trained and setting model parameters of the model to be trained; obtaining multiple historical site pictures to construct a historical sample set, annotating the historical sample set to obtain historical annotated samples; performing data augmentation processing on the historical annotated samples to obtain historical augmented samples; dividing the historical augmented samples, and inputting the divided historical augmented samples into the model to be trained for training to obtain an express delivery recognition model; obtaining real-time images to be recognized, and preprocessing the real-time images to be recognized to obtain real-time preprocessed images; inputting the real-time preprocessed images into the express delivery recognition model to obtain recognition results, and judging whether there is a problem of missing express delivery according to the recognition results.

[0007] Optionally, in a first implementation manner of the first aspect of the present invention, the building of the model to be trained and setting the model parameters of the model to be trained include: using the EfficientNet network as the backbone network of the model to be trained, the EfficientNet including a first convolutional layer, multiple MBConv convolutional modules and a second convolutional layer connected in sequence; the MBConv convolutional module includes a third convolutional layer, a depthwise separable convolutional module, an SE module, a fourth convolutional layer and an output layer connected in sequence; the SE module includes a global average pooling layer, a first fully connected layer and a second fully connected layer connected in sequence; setting the activation function of the third convolutional layer and the activation function of the depthwise separable convolutional module to a Sigmoid activation function, setting the activation function of the first fully connected layer to a Swish activation function, and setting the activation function of the second fully connected layer to a Sigmoid activation function; adding a BiFPN feature fusion network at the output end of the EfficientNet network to complete the building of the model to be trained.

[0008] Optionally, in a second implementation method of the first aspect of the present invention, obtaining multiple historical site pictures to construct a historical sample set, annotating the historical sample set, and obtaining historical annotated samples includes: obtaining multiple historical site pictures, and performing normalization processing and image enhancement processing on the multiple historical site pictures respectively to construct a historical sample set; inputting the historical sample set into a pre-built CNN convolutional neural network model to obtain a region recognition result, wherein the region recognition result includes multiple historical recognition images, each historical recognition image is annotated with one or more recognition boxes; configuring a category label for the recognition box on the historical recognition image, integrating all historical recognition images after the category label configuration is completed, and obtaining historical annotated samples.

[0009] Optionally, in a third implementation manner of the first aspect of the present invention, the data augmentation processing is performed on the historical annotated samples to obtain historical augmented samples, including: dividing the historical augmented samples into a first sample to be augmented, a second sample to be augmented and an original sample according to a preset augmentation division ratio; performing augmentation processing on the first sample to be augmented using a temporal random move method to obtain a first augmented sample; performing augmentation processing on the second sample to be augmented using a random crop method to obtain a second augmented sample; and integrating the first augmented sample, the second augmented sample and the original sample to obtain the historical augmented sample.

[0010] Optionally, in a fourth implementation method of the first aspect of the present invention, the historical augmented samples are divided and processed, and the divided historical augmented samples are input into the model to be trained for training to obtain an express delivery recognition model, including: dividing the historical augmented samples into a training set and a verification set according to a preset training division ratio; inputting the training set into the model to be trained for training, and during the training process, updating the parameters of the model to be trained by an optimization algorithm; inputting the verification set into the trained model to be trained for verification, and during the verification process, adjusting the parameters of the model to be trained according to the evaluation results of the verification set to obtain an express delivery recognition model.

[0011] Optionally, in the fifth implementation method of the first aspect of the present invention, the real-time image to be identified is obtained, and the real-time image to be identified is preprocessed to obtain the real-time preprocessed image, including: when a nuclear order request is received, the real-time image to be identified is obtained; the real-time image to be identified is grayscaled to convert the real-time image to be identified into RGB format; the NumPy library is used to perform array operations on the real-time image to be identified to convert the real-time image to be identified in RGB format into floating point format; and the image transformation function is used to normalize the real-time image to be identified to obtain the real-time preprocessed image.

[0012] Optionally, in a sixth implementation method of the first aspect of the present invention, the real-time pre-processed image is input into the express delivery recognition model to obtain a recognition result, and whether there is a missing express delivery problem is determined based on the recognition result, including: inputting the real-time pre-processed image into the express delivery recognition model to obtain a recognition result, wherein the recognition result is a recognition image including one or more prediction boxes; using a non-maximum suppression algorithm to process the recognition image, and counting the number of prediction boxes in the processed recognition image; obtaining the number of items to be reviewed corresponding to the review request, and if the counted number of prediction boxes is greater than the number of items to be reviewed, it is determined that there is a missing express delivery problem.

[0013] The second aspect of the present invention provides a device for identifying leftover items, including: a building module, used to build a model to be trained and set model parameters of the model to be trained; a labeling module, used to obtain multiple historical site pictures to construct a historical sample set, and label the historical sample set to obtain historical labeled samples; an augmentation module, used to perform data augmentation processing on the historical labeled samples to obtain historical augmented samples; an identification module, used to divide the historical augmented samples, and input the divided historical augmented samples into the model to be trained for training to obtain an express delivery recognition model; a processing module, used to obtain a real-time image to be recognized, and pre-process the real-time image to be recognized to obtain a real-time pre-processed image; a judgment module, used to input the real-time pre-processed image into the express delivery recognition model to obtain a recognition result, and judge whether there is a problem of missing express delivery according to the recognition result.

[0014] Optionally, in a first implementation manner of the second aspect of the present invention, the building module includes: a building unit, used to use the EfficientNet network as the backbone network of the model to be trained, the EfficientNet includes a first convolutional layer, multiple MBConv convolutional modules and a second convolutional layer connected in sequence; the MBConv convolutional module includes a third convolutional layer, a depth-wise separable convolutional module, an SE module, a fourth convolutional layer and an output layer connected in sequence; the SE module includes a global average pooling layer, a first fully connected layer and a second fully connected layer connected in sequence; a setting unit, used to set the activation function of the third convolutional layer and the activation function of the depth-wise separable convolutional module to the Sigmoid activation function, set the activation function of the first fully connected layer to the Swish activation function, and set the activation function of the second fully connected layer to the Sigmoid activation function; an adding unit, used to add a BiFPN feature fusion network at the output end of the EfficientNet network to complete the construction of the model to be trained.

[0015] Optionally, in a second implementation of the second aspect of the present invention, the labeling module includes: a first acquisition unit, used to acquire multiple historical site pictures, and perform normalization and image enhancement processing on the multiple historical site pictures respectively to construct a historical sample set; a first identification unit, used to input the historical sample set into a pre-built CNN convolutional neural network model to obtain a region recognition result, wherein the region recognition result includes multiple historical recognition images, each of which is annotated with one or more recognition boxes; a configuration unit, used to configure a category label for the recognition box on the historical recognition image, and integrate all historical recognition images after the category label configuration is completed to obtain a historical annotated sample.

[0016] Optionally, in a third implementation of the second aspect of the present invention, the augmentation module includes: a first division unit, used to divide the historical augmented samples into first samples to be augmented, second samples to be augmented and original samples according to a preset augmentation division ratio; a first augmentation unit, used to perform augmentation processing on the first samples to be augmented using a temporal random move method to obtain a first augmented sample; a second augmentation unit, used to perform augmentation processing on the second samples to be augmented using a random crop method to obtain a second augmented sample; and an integration unit, used to integrate the first augmented samples, the second augmented samples and the original samples to obtain the historical augmented samples.

[0017] Optionally, in a fourth implementation of the second aspect of the present invention, the recognition module includes: a second division unit, used to divide the historical augmented samples into a training set and a verification set according to a preset training division ratio; a training unit, used to input the training set into the model to be trained for training, and during the training process, update the parameters of the model to be trained by an optimization algorithm; a verification unit, used to input the verification set into the trained model to be trained for verification, and during the verification process, adjust the parameters of the model to be trained according to the evaluation results of the verification set to obtain the express recognition model.

[0018] Optionally, in a fifth implementation of the second aspect of the present invention, the processing module includes: a second acquisition unit, used to acquire a real-time image to be identified when a nuclear order request is received; a first conversion unit, used to perform grayscale processing on the real-time image to be identified to convert the real-time image to be identified into RGB format; a second conversion unit, used to perform array operations on the real-time image to be identified using the NumPy library to convert the real-time image to be identified in RGB format into a floating point format; a first processing unit, used to perform normalization processing on the real-time image to be identified using an image transformation function to obtain a real-time preprocessed image.

[0019] Optionally, in a sixth implementation of the second aspect of the present invention, the judgment module includes: a second recognition unit, used to input the real-time preprocessed image into the express recognition model to obtain a recognition result, wherein the recognition result is a recognition image including one or more prediction boxes; a second processing unit, used to process the recognition image using a non-maximum suppression algorithm, and to count the number of prediction boxes in the processed recognition image; a judgment unit, used to obtain the number of items to be reviewed corresponding to the review request, and if the number of prediction boxes counted is greater than the number of items to be reviewed, it is determined that there is a problem of missing express items.

[0020] A third aspect of the present invention provides a legacy piece identification device, which includes: a memory and at least one processor, wherein the memory stores instructions; at least one of the processors calls the instructions in the memory to enable the legacy piece identification device to perform each step of the legacy piece identification method described in any one of the above items.

[0021] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, and the instructions, when executed by a processor, implement the various steps of any of the above-mentioned legacy parts identification methods.

[0022] In the technical solution of the present invention, a model to be trained is built and model parameters of the model to be trained are set; multiple historical site pictures are obtained and annotated and augmented respectively to obtain historical augmented samples; the historical augmented samples after division are input into the model to be trained for training to obtain an express recognition model; real-time images to be recognized are obtained and preprocessed to obtain real-time preprocessed images; the real-time preprocessed images are input into the express recognition model to obtain recognition results, and whether there is a problem of missing express delivery is determined based on the recognition results; the method disclosed in the present application can realize automatic recognition of express delivery missing problems, reduce the tediousness and errors of manual inspection, ensure the accuracy and timeliness of express delivery processing, effectively improve the management efficiency and quality of outlets, and enhance the logistics experience of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A first flow chart of a legacy component identification method provided by an embodiment of the present invention;

[0024] Figure 2 A second flow chart of the legacy parts identification method provided by an embodiment of the present invention;

[0025] Figure 3 A third flow chart of the legacy component identification method provided by an embodiment of the present invention;

[0026] Figure 4 A fourth flow chart of the legacy component identification method provided by an embodiment of the present invention;

[0027] Figure 5 A fifth flow chart of the legacy parts identification method provided by the embodiment of the present invention;

[0028] Figure 6 A sixth flow chart of the legacy component identification method provided by an embodiment of the present invention;

[0029] Figure 7 A seventh flow chart of the legacy component identification method provided by the embodiment of the present invention;

[0030] Figure 8 A schematic diagram of the structure of a legacy component identification device provided by an embodiment of the present invention;

[0031] Fig. 9 A schematic diagram of the structure of a legacy part identification device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention provides a legacy part identification method, device, equipment and storage medium. In the present invention, the terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the method for identifying a legacy component in an embodiment of the present invention includes:

[0034] 101. Build the model to be trained and set the model parameters of the model to be trained;

[0035] In this embodiment, by building the model to be trained, a solid foundation is laid for the subsequent accurate recognition work, and by adjusting the model parameters of the model to be trained, it is ensured that the model to be trained can better adapt to the recognition scenario of the number of express parcels.

[0036] 102. Acquire multiple historical site images to construct a historical sample set, and annotate the historical sample set to obtain historical annotated samples;

[0037] In this embodiment, a comprehensive historical sample set is constructed by collecting multiple pictures of historical sites, and then the collected pictures of historical sites are labeled to ensure the accuracy and reliability of the sample data, thereby providing high-quality learning materials for the model to be trained.

[0038] 103. Perform data augmentation processing on the historical annotated samples to obtain historical augmented samples;

[0039] In this embodiment, more training samples are generated by rotation, scaling, cropping and other methods to perform data augmentation processing on historically labeled samples, which can increase the diversity of samples and effectively improve the generalization ability of the trained model to be identified for express delivery identification in different scenarios.

[0040] 104. Divide the historical augmented samples, and input the divided historical augmented samples into a to-be-trained model for training to obtain an express delivery recognition model;

[0041] In this embodiment, the augmented samples are divided to obtain a training set and a validation set; the division process ensures that the model to be trained can be effectively evaluated and adjusted during the training process, thereby improving the training effect of the model to be trained, so that the trained express recognition model can accurately identify express parcels in real-time images, that is, it has a high recognition ability and robustness.

[0042] 105. Acquire a real-time image to be recognized, and preprocess the real-time image to be recognized to obtain a real-time preprocessed image;

[0043] In this embodiment, the real-time image to be identified is preprocessed, including denoising, contrast enhancement and other operations, to reduce the interference of environmental noise, ensure image quality, and improve the recognition accuracy and efficiency of the express identification model for the real-time image to be identified.

[0044] 106. Input the real-time pre-processed image into the express delivery recognition model to obtain the recognition result, and determine whether there is a problem of missing express delivery based on the recognition result.

[0045] The present application discloses a method for identifying leftover items, which comprises building a model to be trained and setting model parameters of the model to be trained; obtaining multiple historical site pictures and performing labeling and augmentation processing on them respectively to obtain historical augmented samples; inputting the divided historical augmented samples into the model to be trained to obtain an express item recognition model; obtaining a real-time image to be recognized and performing preprocessing to obtain a real-time preprocessed image; inputting the real-time preprocessed image into the express item recognition model to obtain a recognition result, and judging whether there is a problem of missing express items based on the recognition result; the method disclosed in the present application can realize automatic recognition of the problem of missing express items, reduce the tediousness and errors of manual inspection, ensure the accuracy and timeliness of express item processing, effectively improve the management efficiency and quality of outlets, and enhance the logistics experience of users.

[0046] See also Figure 2 , a second embodiment of the method for identifying a legacy component in an embodiment of the present invention includes:

[0047] 201. An EfficientNet network is used as a backbone network of a model to be trained, wherein the EfficientNet network includes a first convolutional layer, a plurality of MBConv convolutional modules, and a second convolutional layer connected in sequence; the MBConv convolutional module includes a third convolutional layer, a depthwise separable convolutional module, an SE module, a fourth convolutional layer, and an output layer connected in sequence; the SE module includes a global average pooling layer, a first fully connected layer, and a second fully connected layer connected in sequence;

[0048] In this embodiment, the first convolution layer and the second convolution layer are 3×3 convolution layers, and the third convolution layer and the fourth convolution layer are 1×1 convolution layers.

[0049] Set the activation function of the third convolutional layer and the activation function of the depthwise separable convolutional module to the Sigmoid activation function, set the activation function of the first fully connected layer to the Swish activation function, and set the activation function of the second fully connected layer to the Sigmoid activation function;

[0050] In this embodiment, the EfficientNet network is the EfficientNet-B0 network. The main part of the network is mainly divided into two parts: the first layer and the last layer use ordinary convolutions with a kernel size of 3×3, and the remaining layers use repeatedly stacked MBConv convolutional modules; the MBConv convolutional module contains 5 main operations, which are: 1×1 convolution for dimensionality increase. The input feature map is convolved by 1×1 convolution, and the number of convolution kernels is n (1 < n < 6) times the number of channels of the input feature map, so as to increase the number of channels; the depthwise separable convolutional module, which contains two steps of depth convolution and pointwise convolution for feature extraction; the SE attention mechanism module for calculating the importance weights of features to enhance the response of important features; 1×1 convolution for dimensionality reduction to adjust the dimension to the number of output channels; the Dropout layer randomly discards to reduce overfitting; in the MBConv convolutional module, BN layers and Swish activation functions are introduced in both the 1×1 convolution layer for dimensionality increase and the depthwise separable convolutional module, and the SE module is added; the SE module consists of a global average pooling, two fully connected layers and an activation function; first, the global average pooling compresses the input feature map to obtain the average value of each channel, then reduces the dimension through the first fully connected layer and the Swish activation function, and then restores the dimension through the second fully connected layer and the Sigmoid activation function to obtain the weight of each channel. Finally, multiply the weight by the original feature map to implement the attention mechanism between channels; the SE module can dynamically adjust the importance of each channel in the feature map, thereby improving the expression ability of the network.

[0051] 203. Add a BiFPN feature fusion network at the output end of the EfficientNet network to complete the construction of the model to be trained;

[0052] In this embodiment, the BiFPN feature fusion network, as an extension of the EfficientNet backbone network, undertakes the task of efficiently fusing the five initially extracted feature maps of different scales; this network architecture is innovative on the basis of FPN, and by introducing effective bidirectional cross connections, it realizes top-down information flow and bottom-up feature integration, so that the entire network can more keenly capture key hierarchical information; further, BiFPN optimizes the feature selection process through a weighted feature fusion mechanism, reduces the dependence on less important layers, and thus improves the overall computational efficiency; further, BiFPN adds an innovative edge to the horizontal connection, which directly connects the input node with the output node to form a structure similar to a residual connection. The residual connection-like structure not only does not increase the additional computational burden, but enables the network to fuse more complex and rich features, further improving the performance of the model; when processing multi-scale feature fusion, the BiFPN feature fusion network maintains high efficiency and ensures the depth and breadth of feature fusion, providing strong support for the realization of complex visual tasks.

[0053] See also Figure 3 , a third embodiment of the method for identifying legacy parts in the embodiment of the present invention includes:

[0054] 301. Acquire multiple historical site images, and perform normalization and image enhancement processing on the multiple historical site images respectively to construct a historical sample set;

[0055] In this embodiment, in order to ensure the diversity of the multiple historical site images obtained and to ensure that the samples can fully cover all areas and characteristics of the site, images of the logistics site can be collected at different time periods and under different lighting conditions, and photographed from multiple angles, including overhead and side shots, to obtain more comprehensive express appearance information.

[0056] In this embodiment, by performing normalization processing and image enhancement processing on multiple historical site pictures respectively, the sizes of the multiple historical site pictures can be ensured to be consistent, the picture quality can be improved, and the training effect of the model can be enhanced.

[0057] 302. Input the historical sample set into a pre-built CNN convolutional neural network model to obtain a region recognition result, wherein the region recognition result includes a plurality of historical recognition images, each of which is marked with one or more recognition boxes;

[0058] In this embodiment, by pre-building a CNN convolutional neural network for key area recognition in an image, that is, for express area recognition in an image, the recognition accuracy and efficiency can be effectively improved.

[0059] 303. A category label is configured for the recognition box on the historical recognition image, and all the historical recognition images after the category label configuration are integrated to obtain historical annotated samples;

[0060] In this embodiment, in the historical recognition image, each recognition frame corresponds to a package, and a package label can be marked on each recognition frame to complete the marking of the historical recognition image.

[0061] See also Figure 4 , a fourth embodiment of the method for identifying a legacy component in an embodiment of the present invention includes:

[0062] 401. According to a preset augmentation division ratio, the historical augmented samples are divided into a first sample to be augmented, a second sample to be augmented, and an original sample;

[0063] In this embodiment, the preset augmentation division ratio may be 4:4:2, that is, 2 / 5 of the historical augmented samples are used as the first samples to be augmented, 2 / 5 of the historical augmented samples are used as the second samples to be augmented, and 1 / 5 of the historical augmented samples are used as the original samples.

[0064] 402. Augment the first sample to be augmented by using a temporal random move method to obtain a first augmented sample.

[0065] In this embodiment, temporal random move involves randomly moving frames in time series data. This operation can simulate the change of data in time and provide a more flexible and variable input for the model. For example, randomly moving an image by a certain number of pixels in the horizontal or vertical direction can simulate the possible position change of the express at different time points. If there are only static images, the time change can be simulated by copying the image and randomly moving it in the time dimension. For example, copying an image multiple times, and then performing different degrees of translation operations on each copied image, and finally combining these images into a data set similar to a video sequence.

[0066] 403. Using a random crop method to perform augmentation processing on the second sample to be augmented, to obtain a second augmented sample;

[0067] In this embodiment, random crop operates on the data in the spatial dimension, creating new training samples by randomly cropping a portion of an image or video frame. This method can force the model to focus on the key parts of the image instead of relying on a specific background or scene layout; for example, the side length of the cropped area can be randomly selected between 50% and 80% of the side length of the original image, ensuring that at least one express delivery is included in the cropped area to ensure the validity of the data; a express delivery can be randomly selected from the annotated data, and then the position and size of the cropped area can be determined with the express delivery as the center.

[0068] 404. Integrate the first augmented sample, the second augmented sample, and the original sample to obtain a historical augmented sample.

[0069] See also Figure 5 The fifth embodiment of the method for identifying a legacy component in the embodiment of the present invention includes:

[0070] 501. Divide the historical augmented samples into a training set and a validation set according to a preset training division ratio;

[0071] In this embodiment, 80% of the historical augmented samples can be used as training sets, and 20% of the historical augmented samples can be used as validation sets; further, the historical augmented samples can be divided using a random division method to ensure that the data distribution of the training set and the validation set is representative.

[0072] 502. Input the training set into the model to be trained for training. During the training process, update the parameters of the model to be trained by using an optimization algorithm.

[0073] In this embodiment, in each training batch, the image data and the corresponding annotation information are input into the model, and the loss function between the model's prediction results and the actual annotations is calculated; an optimization algorithm (such as the Adam optimizer) can be used to update the model's parameters to minimize the loss function; the model parameters are saved every certain number of training rounds, for example, once every 10 rounds.

[0074] 503. Input the verification set into the trained model to be trained for verification. During the verification process, adjust the parameters of the model to be trained according to the evaluation results of the verification set to obtain the express delivery recognition model.

[0075] In this embodiment, the validation set is input into the model to be trained, and the accuracy, recall and other indicators of the model on the validation set are calculated to monitor the performance of the model. According to the evaluation results of the validation set, the parameters or training strategy of the model are adjusted. If the performance of the model on the validation set no longer improves, you can consider reducing the learning rate, increasing the number of training rounds, or adjusting the data augmentation method.

[0076] See also Figure 6 The sixth embodiment of the method for identifying legacy parts in the embodiment of the present invention includes:

[0077] 601. When receiving a verification request, obtain a real-time image to be recognized;

[0078] In this embodiment, when the courier completes picking up the batch of goods and is ready to leave the distribution center, a request for order verification can be sent to determine whether there are any missing items, thereby reducing the possibility of missing items.

[0079] 602. Perform grayscale processing on the real-time image to be recognized to convert the real-time image to be recognized into RGB format;

[0080] 603. Use the NumPy library to perform array operations on the real-time image to be recognized, so as to convert the real-time image to be recognized in RGB format into a floating point format;

[0081] In this embodiment, before normalization, it is usually necessary to convert the image from color to grayscale, or to RGB format, depending on the format used during training; the conversion to RGB format can be achieved through OpenCV's `cv2.cvtColor` function. Next, use the NumPy library for array operations to convert the image data into floating-point format to facilitate subsequent normalization processing.

[0082] 604. Performing normalization processing on the real-time image to be recognized using an image transformation function to obtain a real-time preprocessed image;

[0083] In this embodiment, in order to ensure that the image data is consistent with the training stage, the captured image needs to be adjusted to the same size as that during training; the size conversion can be achieved by using an image processing library such as OpenCV; in addition, in order to further process the image data, the image needs to be normalized; the purpose of normalization is to scale the pixel values ​​of the image data to a uniform range, usually 0 to 1 or -1 to 1, so that the model can better process and learn.

[0084] See also Figure 7 The seventh embodiment of the method for identifying legacy parts in the embodiment of the present invention includes:

[0085] 701. Input the real-time preprocessed image into the express delivery recognition model to obtain a recognition result, where the recognition result is a recognition image including one or more prediction boxes;

[0086] 702. Process the recognition image using a non-maximum suppression algorithm, and count the number of prediction boxes in the processed recognition image;

[0087] In an embodiment, by applying a non-maximum suppression algorithm to perform detailed processing on the recognition image, those prediction boxes with low confidence levels can be effectively removed, thereby improving the accuracy of image recognition; the core of the non-maximum suppression algorithm is to identify and eliminate those candidate boxes with low confidence levels, and at the same time effectively merge those prediction boxes with high overlap to ensure that each target object in the final result corresponds to only one most accurate prediction box, which not only improves the recognition accuracy, but also optimizes the display of the results, so that the target objects in the image can be marked more clearly and accurately.

[0088] 703. Obtain the number of items to be approved corresponding to the approval request. If the number of predicted boxes counted is greater than the number of items to be approved, it is determined that there is a problem of missing items.

[0089] In this embodiment, the number to be reviewed can be calculated based on the real-time total number of pieces in the distribution center and the number of pieces picked up by the courier; when the number of predicted boxes counted, that is, the number of express parcels counted is greater than the number to be reviewed, there is a problem of missing express parcels, and an early warning can be issued at this time; the early warning method can include sending e-mails, text message notifications, displaying alarm information on the monitoring interface, etc.; Python's mail sending library (such as smtplib) or text message sending library can be used to implement early warning notifications; at the same time, the location of the remaining express parcels can be marked with eye-catching colors on the monitoring interface so that the staff can find and handle them in time.

[0090] The above describes the method for identifying a leftover item in an embodiment of the present invention. The following describes the device for identifying a leftover item in an embodiment of the present invention. Figure 8 , an embodiment of the device for identifying a legacy part in an embodiment of the present invention includes:

[0091] A building module 801 is used to build a model to be trained and set model parameters of the model to be trained; a labeling module 802 is used to obtain multiple historical site pictures to construct a historical sample set, and label the historical sample set to obtain historical labeled samples; an augmentation module 803 is used to perform data augmentation on the historical labeled samples to obtain historical augmented samples; an identification module 804 is used to divide the historical augmented samples, and input the divided historical augmented samples into the model to be trained for training to obtain an express delivery recognition model; a processing module 805 is used to obtain a real-time image to be recognized, and pre-process the real-time image to be recognized to obtain a real-time pre-processed image; a judgment module 806 is used to input the real-time pre-processed image into the express delivery recognition model to obtain a recognition result, and judge whether there is a problem of missing express delivery according to the recognition result.

[0092] In this embodiment, the building module 801 includes: a building unit 8011, which is used to use the EfficientNet network as the backbone network of the model to be trained, and the EfficientNet includes a first convolutional layer, multiple MBConv convolutional modules and a second convolutional layer connected in sequence; the MBConv convolutional module includes a third convolutional layer, a depth-wise separable convolutional module, an SE module, a fourth convolutional layer and an output layer connected in sequence; the SE module includes a global average pooling layer, a first fully connected layer and a second fully connected layer connected in sequence; a setting unit 8012, which is used to set the activation function of the third convolutional layer and the activation function of the depth-wise separable convolutional module to the Sigmoid activation function, set the activation function of the first fully connected layer to the Swish activation function, and set the activation function of the second fully connected layer to the Sigmoid activation function; an additional unit 8013, which is used to add a BiFPN feature fusion network at the output end of the EfficientNet network to complete the construction of the model to be trained.

[0093] In this embodiment, the labeling module 802 includes: a first acquisition unit 8021, which is used to acquire multiple historical site pictures, and perform normalization and image enhancement processing on the multiple historical site pictures respectively to construct a historical sample set; a first recognition unit 8022, which is used to input the historical sample set into a pre-built CNN convolutional neural network model to obtain a region recognition result, and the region recognition result includes multiple historical recognition images, each of which is marked with one or more recognition boxes; a configuration unit 8023, which is used to configure a category label for the recognition box on the historical recognition image, integrate all historical recognition images after the category label configuration is completed, and obtain a historical labeled sample.

[0094] In this embodiment, the augmentation module 803 includes: a first division unit 8031, which is used to divide the historical augmented samples into first samples to be augmented, second samples to be augmented and original samples according to a preset augmentation division ratio; a first augmentation unit 8032, which is used to perform augmentation processing on the first samples to be augmented using a temporal random move method to obtain a first augmented sample; a second augmentation unit 8033, which is used to perform augmentation processing on the second samples to be augmented using a random crop method to obtain a second augmented sample; and an integration unit 8034, which is used to integrate the first augmented samples, the second augmented samples and the original samples to obtain the historical augmented samples.

[0095] In this embodiment, the recognition module 804 includes: a second division unit 8041, which is used to divide the historical augmented samples into a training set and a verification set according to a preset training division ratio; a training unit 8042, which is used to input the training set into the model to be trained for training, and during the training process, update the parameters of the model to be trained by an optimization algorithm; a verification unit 8043, which is used to input the verification set into the trained model to be trained for verification, and during the verification process, adjust the parameters of the model to be trained according to the evaluation results of the verification set to obtain the express recognition model.

[0096] In this embodiment, the processing module 805 includes: a second acquisition unit 8051, which is used to acquire the real-time image to be recognized when a nuclear order request is received; a first conversion unit 8052, which is used to perform grayscale processing on the real-time image to be recognized to convert the real-time image to be recognized into RGB format; a second conversion unit 8053, which is used to use the NumPy library to perform array operations on the real-time image to be recognized to convert the real-time image to be recognized in RGB format into a floating point format; a first processing unit 8054, which is used to use an image transformation function to normalize the real-time image to be recognized to obtain a real-time preprocessed image.

[0097] In this embodiment, the judgment module 806 includes: a second recognition unit 8061, which is used to input the real-time pre-processed image into the express recognition model to obtain a recognition result, and the recognition result is a recognition image including one or more prediction boxes; a second processing unit 8062, which is used to process the recognition image using a non-maximum suppression algorithm, and count the number of prediction boxes in the processed recognition image; a judgment unit 8063, which is used to obtain the number of items to be reviewed corresponding to the review request. If the number of prediction boxes counted is greater than the number of items to be reviewed, it is determined that there is a missing express problem.

[0098] Based on the same idea as the method in the above embodiment, the device provided by the present application can implement the method in the above embodiment.

[0099] above Figure 8 The legacy component identification device in the embodiment of the present invention is described in detail from the perspective of modular functional entities, and the legacy component identification device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0100] Fig. 91 is a schematic diagram of the structure of a legacy piece identification device provided by an embodiment of the present invention. The legacy piece identification device 900 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 910 (for example, one or more processors) and a memory 920, and one or more storage media 930 (for example, one or more mass storage devices) storing application programs 933 or data 932. Among them, the memory 920 and the storage medium 930 can be temporary storage or permanent storage. The program stored in the storage medium 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the legacy piece identification device 900. Furthermore, the processor 910 can be configured to communicate with the storage medium 930, and execute a series of instruction operations in the storage medium 930 on the legacy piece identification device 900 to implement the steps of the legacy piece identification method provided by the above-mentioned method embodiments.

[0101] The legacy identification device 900 may also include one or more power supplies 940, one or more wired or wireless network interfaces 950, one or more input and output interfaces 960, and / or one or more operating systems 931, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Fig. 9 The structure of the legacy part identification device shown does not constitute a limitation on the legacy part identification device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0102] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the legacy component identification method.

[0103] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0104] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0105] Finally, it should be noted that the above description is only a preferred example of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying legacy parts, characterized in that: include: Build the model to be trained and set the model parameters of the model to be trained; Acquire multiple historical site images to construct a historical sample set, and annotate the historical sample set to obtain historical annotated samples; Perform data augmentation processing on historical annotated samples to obtain historical augmented samples; The historical augmented samples are divided and processed, and the divided historical augmented samples are input into the model to be trained for training to obtain an express delivery recognition model; Acquire a real-time image to be recognized, and preprocess the real-time image to be recognized to obtain a real-time preprocessed image; The real-time preprocessed image is input into the express delivery recognition model to obtain the recognition result, and it is determined whether there is a problem of missing express delivery based on the recognition result.

2. The method for identifying legacy parts according to claim 1, characterized in that: The step of building a model to be trained and setting model parameters of the model to be trained includes: The EfficientNet network is used as the backbone network of the model to be trained, wherein the EfficientNet network includes a first convolutional layer, multiple MBConv convolutional modules, and a second convolutional layer connected in sequence; the MBConv convolutional module includes a third convolutional layer, a depthwise separable convolutional module, an SE module, a fourth convolutional layer, and an output layer connected in sequence; the SE module includes a global average pooling layer, a first fully connected layer, and a second fully connected layer connected in sequence; Set the activation function of the third convolutional layer and the activation function of the depthwise separable convolutional module to the Sigmoid activation function, set the activation function of the first fully connected layer to the Swish activation function, and set the activation function of the second fully connected layer to the Sigmoid activation function; A BiFPN feature fusion network is added to the output end of the EfficientNet network to complete the construction of the model to be trained.

3. The method for identifying legacy parts according to claim 1, characterized in that: The step of obtaining a plurality of historical site images to construct a historical sample set and annotating the historical sample set to obtain historical annotated samples includes: Acquire multiple historical site pictures, and perform normalization and image enhancement processing on the multiple historical site pictures respectively to construct a historical sample set; Inputting the historical sample set into a pre-built CNN convolutional neural network model to obtain a region recognition result, wherein the region recognition result includes a plurality of historical recognition images, each of which is marked with one or more recognition boxes; A category label is configured for each recognition box on the historical recognition image, and all historical recognition images after category label configuration are integrated to obtain historical annotated samples.

4. The method for identifying legacy parts according to claim 1, characterized in that: The data augmentation process is performed on the historically annotated samples to obtain historically augmented samples, including: According to the preset augmentation division ratio, the historical augmented samples are divided into the first to-be-augmented samples, the second to-be-augmented samples and the original samples; The first sample to be augmented is augmented by using the temporal random move method to obtain a first augmented sample; The second sample to be augmented is augmented using a random crop method to obtain a second augmented sample; The first augmented sample, the second augmented sample and the original sample are integrated to obtain the historical augmented sample.

5. The method for identifying legacy parts according to claim 1, characterized in that: The historical augmented samples are divided and processed, and the divided historical augmented samples are input into the model to be trained to obtain the express delivery recognition model, including: According to a preset training division ratio, the historical augmented samples are divided into a training set and a validation set; Inputting the training set into the model to be trained for training, and during the training process, updating the parameters of the model to be trained by an optimization algorithm; The validation set is input into the trained model for verification. During the verification process, the parameters of the model to be trained are adjusted according to the evaluation results of the validation set to obtain the express delivery recognition model.

6. The method for identifying legacy parts according to claim 1, characterized in that: The step of acquiring a real-time image to be recognized and preprocessing the real-time image to be recognized to obtain a real-time preprocessed image includes: When receiving a request for a verification order, obtain the real-time image to be recognized; Grayscale processing is performed on the real-time image to be recognized to convert the real-time image to be recognized into RGB format; Use the NumPy library to perform array operations on the real-time image to be recognized, so as to convert the real-time image to be recognized in RGB format into a floating point format; The real-time image to be recognized is normalized using an image transformation function to obtain a real-time preprocessed image.

7. The method for identifying legacy parts according to claim 6, characterized in that: The real-time pre-processed image is input into the express mail recognition model to obtain a recognition result, and judging whether there is a problem of missing express mail according to the recognition result includes: Inputting the real-time preprocessed image into the express delivery recognition model to obtain a recognition result, wherein the recognition result is a recognition image including one or more prediction boxes; Use the non-maximum suppression algorithm to process the recognition image and count the number of prediction boxes in the processed recognition image; The number of items to be reviewed corresponding to the review request is obtained. If the number of predicted boxes counted is greater than the number to be reviewed, it is determined that there is a missing item problem.

8. A device for identifying a leftover part, characterized in that: include: A building module is used to build the model to be trained and set the model parameters of the model to be trained; The annotation module is used to obtain multiple historical site images to construct a historical sample set, annotate the historical sample set, and obtain historical annotated samples; The augmentation module is used to perform data augmentation processing on historical annotated samples to obtain historical augmented samples; An identification module, used for dividing the historical augmented samples, and inputting the divided historical augmented samples into a to-be-trained model for training, so as to obtain an express identification model; A processing module is used to obtain a real-time image to be identified and preprocess the real-time image to be identified to obtain a real-time preprocessed image; The judgment module is used to input the real-time pre-processed image into the express delivery recognition model to obtain the recognition result, and judge whether there is a problem of missing express delivery based on the recognition result.

9. A device for identifying a legacy part, characterized in that: The legacy piece identification device comprises: a memory and at least one processor, wherein instructions are stored in the memory; At least one of the processors calls the instructions in the memory to enable the legacy piece identification device to perform each step of the legacy piece identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the various steps of the legacy component identification method as described in any one of claims 1-7 are implemented.

Images