Second-hand commodity double-track grading evaluation method and system based on deep learning

Abstract

The invention relates to the technical field of second-hand evaluation, and discloses a second-hand commodity double-track grading evaluation method and system based on deep learning, and the method comprises the steps: collecting image data and attribute data of a second-hand commodity, and carrying out the preprocessing of the image data and attribute data, and obtaining the preprocessed image data and attribute data; inputting the preprocessed image data and attribute data into a double-track multi-modal model, and extracting a visual feature vector of a second-hand commodity by an image processing track through a ResNet architecture; the text processing track performs feature coding on the commodity description text through an embedded layer and a hybrid encoder to obtain an attribute feature vector; performing dynamic weighted fusion on the visual feature vector and the attribute feature vector through an attention fusion mechanism, inputting the fused feature vector into a full connection layer, and outputting a grading result of the commodity; according to the invention, automation of second-hand commodity grading evaluation is realized, the evaluation efficiency is greatly improved, and the labor cost is reduced.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of second-hand evaluation, and particularly relates to a second-hand commodity double-track grading evaluation method and system based on deep learning. BACKGROUND

[0002] With the vigorous development of the second-hand commodity trading market, fast and accurate grading evaluation of second-hand commodities has become a key link; at present, grading evaluation of second-hand commodities mainly depends on manual identification, and the evaluation result is greatly influenced by the experience and subjective judgment of the identification personnel, and there are problems such as non-uniform standards, low efficiency and insufficient accuracy. Some existing automatic evaluation methods are based on single features for evaluation, such as judging the grade only by identifying the appearance of the commodity through image recognition, ignoring important non-visual attribute information such as the brand, service life and repair record of the commodity, resulting in one-sided evaluation results; meanwhile, when dealing with complex commodity features, traditional machine learning algorithms are difficult to capture deep feature correlations, and the evaluation accuracy is limited. SUMMARY

[0003] The application aims to solve the above problems, and designs a second-hand commodity double-track grading evaluation method and system based on deep learning.

[0004] The first aspect of the application provides a second-hand commodity double-track grading evaluation method based on deep learning, which comprises the following steps: Collecting image data and attribute data of second-hand commodities and respectively pre-processing to obtain pre-processed image data and attribute data; Inputting the pre-processed image data and attribute data into a double-track multi-modal model, wherein the double-track multi-modal model comprises an image processing track and a text processing track; The pre-processed image data enters the image processing track, and the image processing track extracts a visual feature vector of the second-hand commodity through a ResNet architecture; The pre-processed attribute data enters the text processing track, and the text processing track encodes the feature of the commodity description text through an embedding layer and a hybrid encoder to obtain an attribute feature vector; The visual feature vector and the attribute feature vector are dynamically weighted and fused through an attention fusion mechanism, the fused feature vector is input into a full connection layer, and a grading result of the commodity is output.

[0005] Optionally, in the first implementation manner of the first aspect of the application, the collecting of the image data and the attribute data of the second-hand commodities and the pre-processing to obtain the pre-processed image data and the attribute data comprises: A U-Net model is used to extract a commodity main body region from the image data of the second-hand commodity, and background interference is removed; The data enhancement processing is performed through random rotation, horizontal flip, brightness disturbance and Gaussian noise, and image denoising, size normalization and color correction processing are performed to obtain preprocessed image data.

[0006] Optionally, in the second implementation form of the first aspect of the present application, the U-Net model is used to extract the main body region of the second-hand commodity from the image data of the second-hand commodity, and remove background interference, comprising: The image of the second-hand commodity to be processed is input into the U-Net model, a plurality of convolution kernels are used to extract local features of the image, and then a maximum pooling operation is used to select the maximum value of the local region, gradually reducing the spatial size of the feature map and increasing the number of feature channels, to obtain deep semantic features of the image; The image of the second-hand commodity to be processed is fused with the deep semantic features through a skip connection to form a fused feature map; The fused feature map is input into the output layer of the U-Net model, and a convolution operation is performed through the same convolution kernel as the main body and the background of the commodity, and a softmax activation function is combined to obtain the probability distribution of each pixel belonging to the main body or the background of the commodity; According to the preset probability threshold, the output probability distribution is judged to distinguish the background and the main body of the commodity, and the extraction of the main body region of the commodity is completed.

[0007] Optionally, in the third implementation form of the first aspect of the present application, the image data and attribute data of the second-hand commodity are collected and preprocessed to obtain preprocessed image data and attribute data, and further comprising: The attribute data of the second-hand commodity is detected and outliers are removed by using the IQR method, and the missing values in the attribute data are filled by using the KNN algorithm to obtain cleaned attribute data; The Word2Vec model is used to convert the category type data in the cleaned attribute data into a 32-dimensional vector, and the time series data in the cleaned attribute data is sorted according to the timestamp; The numerical value type data in the cleaned attribute data is subjected to Z-score standardization to obtain preprocessed attribute data.

[0008] Optionally, in the fourth implementation form of the first aspect of the present application, the preprocessed image data enters an image processing track, and the image processing track extracts a visual feature vector of the second-hand commodity through a ResNet architecture, comprising: The preprocessed image data is input into the ResNet architecture, a plurality of convolution kernels are used to extract local features of the image in each convolution layer of the backbone network, and then the feature map size is reduced through a maximum pooling operation; The feature maps of corresponding scales are output at different levels of the backbone network and sent to the feature pyramid network, and the 1 / 4, 1 / 8 and 1 / 16 scale feature maps are fused by top-down sampling and horizontal connection to form multi-scale fusion feature maps; The weight of each channel of the fusion feature map is calculated through the channel attention module, and the weight of each spatial position of the fusion feature map is allocated through the spatial attention module, so as to automatically focus on the key region of the second-hand commodity, wherein the key region at least includes a defect position and a serious wear region. The processed feature map continues to be subjected to feature extraction through the convolution layer and the pooling layer, and finally is converted into a fixed-length vector through a global average pooling operation to obtain a visual feature vector.

[0009] Optionally, in the fifth implementation manner of the first aspect of the present application, the preprocessed attribute data enters a text processing track, and the text processing track encodes the feature of the commodity description text through an embedding layer and a hybrid encoder to obtain an attribute feature vector, including: The preprocessed attribute data is divided into category type attributes and numerical type attributes, and the category type attributes are input into the embedding layer to map each category text into a 64-dimensional vector; The category type attribute vector output by the embedding layer is spliced with the numerical type attribute according to the field order to form an initial attribute feature matrix, wherein each row of the attribute feature matrix corresponds to an attribute feature combination; The initial attribute feature matrix is sorted according to the time order to form sequence data input into an LSTM module in the hybrid encoder, and the LSTM module captures the short-term time sequence dependency relationship between attributes through a gating unit; The sequence data is input into a Transformer module in the hybrid encoder, the Transformer module calculates the association weight of each attribute and other attributes through a multi-head self-attention mechanism, captures long-distance feature dependency, and performs nonlinear conversion on the attention output through a feedforward neural network; The output features of the LSTM module and the Transformer module are fused through a gating fusion mechanism to obtain the final attribute feature vector.

[0010] Optionally, in the sixth implementation manner of the first aspect of the present application, the visual feature vector and the attribute feature vector are dynamically weighted and fused through the attention fusion mechanism, the fused feature vector is input into a fully connected layer, and a grading result of the commodity is output, including: The visual feature vector output by the image processing track and the attribute feature vector output by the text processing track are acquired; The visual feature vector and the attribute feature vector are input into a feature interaction module in the attention fusion mechanism, and the correlation score between the two vectors is calculated through a dot product operation. Set the initial weight based on the commodity type classification result, and dynamically adjust the initial weight according to the correlation score; According to the adjusted weight, the visual feature vector and the attribute feature vector are weighted and summed to obtain a fused comprehensive feature vector; The fused comprehensive feature vector is input into a full connection layer, a softmax activation function is applied at the output end of the full connection layer, the output value is converted into a probability distribution of each grade, and the grade with the highest probability is selected as the final grading result of the commodity.

[0011] The second aspect of the application provides a second-hand commodity double-track grading evaluation system based on deep learning, which comprises: A preprocessing module is used to collect image data and attribute data of the second-hand commodity, and the image data and the attribute data are preprocessed respectively to obtain preprocessed image data and attribute data; An input module is used to input the preprocessed image data and attribute data into a double-track multi-modal model, wherein the double-track multi-modal model comprises an image processing track and a text processing track; An extraction module is used to input the preprocessed image data into the image processing track, and the image processing track extracts a visual feature vector of the second-hand commodity through a ResNet architecture; An encoding module is used to input the preprocessed attribute data into the text processing track, and the text processing track encodes the feature of the commodity description text through an embedding layer and a hybrid encoder to obtain an attribute feature vector; A fusion module is used to dynamically weight and fuse the visual feature vector and the attribute feature vector through an attention fusion mechanism, input the fused feature vector into a full connection layer, and output a grading result of the commodity.

[0012] The third aspect of the application provides a second-hand commodity double-track grading evaluation device based on deep learning, which comprises a memory and at least one processor, and the memory stores instructions; the at least one processor calls the instructions in the memory to make the second-hand commodity double-track grading evaluation device based on deep learning execute the steps of the second-hand commodity double-track grading evaluation method based on deep learning according to any one of the above.

[0013] The fourth aspect of the application provides a computer readable storage medium, which stores instructions, and the instructions are executed by a processor to realize the steps of the second-hand commodity double-track grading evaluation method based on deep learning according to any one of the above.

[0014] In the technical scheme provided by the application, the image data and attribute data of the second-hand commodity are collected and preprocessed respectively to obtain preprocessed image data and attribute data; the preprocessed image data and attribute data are input into a double-track multi-modal model, the preprocessed image data enters an image processing track, and the image processing track extracts a visual feature vector of the second-hand commodity through a ResNet architecture; the preprocessed attribute data enters a text processing track, and the text processing track encodes a feature of a commodity description text through an embedding layer and a hybrid encoder to obtain an attribute feature vector; the visual feature vector and the attribute feature vector are dynamically weighted and fused through an attention fusion mechanism, the fused feature vector is input into a full connection layer, and a grading result of the commodity is output; the double-track evaluation architecture is adopted, the visual feature and the attribute feature of the second-hand commodity are considered at the same time, the one-sidedness of single feature evaluation is avoided, the accuracy and comprehensiveness of the evaluation result are improved, the attention fusion mechanism can dynamically adjust the double-track weight according to the type of the commodity, the evaluation is more flexible and meets the actual demand, the automation of the second-hand commodity grading evaluation is realized, the evaluation efficiency is greatly improved, the labor cost is reduced, and an objective and scientific basis is provided for the second-hand commodity transaction. BRIEF DESCRIPTION OF DRAWINGS

[0015] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of preferred embodiments and are not meant to limit the present application.

[0016] Figure 1 A flowchart of a second-hand commodity double-track grading evaluation method based on deep learning provided by an embodiment of the application; Figure 2 A structure schematic diagram of a second-hand commodity double-track grading evaluation system based on deep learning provided by an embodiment of the application; Figure 3 A structure schematic diagram of a second-hand commodity double-track grading evaluation device based on deep learning provided by an embodiment of the application. DETAILED DESCRIPTION

[0017] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, and above-mentioned drawings, if any, are used to distinguish between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so-termed "first", "second", "third", "fourth" and the like, if any, in the description and in the claims of the present application is simply intended to differentiate between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used in the description and in the claims of the present application can be interchanged, where appropriate, so that the embodiments described herein can be carried out in other than the order described or illustrated herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of steps or units not merely those stated in the specification itself, but can include other not expressly listed steps or units inherent therein.

[0018] For the purpose of facilitating understanding, the specific flow of the embodiments of the present application is described below, please refer to Figure 1 The flowchart of the method for evaluating second-hand goods based on deep learning provided by the embodiments of the present application, which specifically includes the following steps: Step 101, collecting image data and attribute data of second-hand goods, and respectively pre-processing to obtain pre-processed image data and attribute data; In this embodiment, when collecting image data of second-hand goods, the front, back, side, top and bottom of the goods are photographed from multiple angles to ensure that the overall appearance and detailed features of the goods are covered, and images are collected under different lighting conditions such as natural light and indoor light to reflect the visual presentation of the goods in various environments; Close-up shots are taken of key parts such as flaws, wear and tear, and scratches on the surface of the goods to clearly record their state information. When collecting attribute data, not only the basic information of the goods such as brand, model, production batch, original price, purchase date, etc. is collected, but also usage-related data such as usage time, usage frequency, maintenance condition, repair record, and performance-related data such as current function integrity, comparison of measured values and standard values of various performance parameters, historical fault records, etc. are collected to form a comprehensive second-hand goods data set.

[0019] In this embodiment, the U-Net model is used to extract the main body region of the second-hand commodity from the image data of the second-hand commodity, and remove the background interference; when the image data is subjected to data enhancement and standardization processing, first, the image is randomly rotated within the range of -15° to 15° to simulate the presentation state of the commodity under different shooting angles; at the same time, the horizontal flip processing is carried out, and the image is mirror flipped along the vertical central axis with a probability of 50%, to expand the visual diversity in the data set; on this basis, the brightness disturbance is implemented, and the brightness value of the image is increased or decreased by a random value within 20% of the original basis to cope with the shooting scene under different light environments; and Gaussian noise with a variance of 0.01 is injected into the image to enhance the resistance of the model to subtle interference. After completing the data enhancement, the image is denoised by using Gaussian filtering to smooth the high-frequency noise points in the image; then all the images are uniformly adjusted to 224x224 pixels in size to ensure that the image specifications of the input model are consistent; finally, the image is converted to the CIELAB color space, and the mean and variance of the brightness channel and the two chroma channels are standardized respectively to eliminate the color deviation caused by different equipment shooting, and finally the preprocessed image data meeting the input requirements of the model is obtained.

[0020] In this embodiment, the image of the second-hand commodity to be processed is input into the U-Net model, and a plurality of convolution kernels are used to extract local features of the image, and then the maximum value of the local region is selected through the maximum pooling operation, the spatial size of the feature map is gradually reduced, and the number of feature channels is increased, to obtain the deep semantic features of the image; the image to be processed and the deep semantic features are fused through the jump connection to form a fused feature map; the fused feature map is input into the output layer of the U-Net model, and convolution operation is performed through the same convolution kernel as the two classes of commodity main body and background, and the probability distribution of each pixel belonging to the commodity main body or the background is obtained through the softmax activation function; according to the preset probability threshold, the output probability distribution is judged to distinguish the background and the commodity main body, and the extraction of the main body region of the commodity is completed.

[0021] In this embodiment, when the IQR method is used to detect and remove outliers of the attribute data of the second-hand commodity, the lower quartile and the upper quartile of each numerical attribute are calculated, and the interquartile range IQR = Q3-Q1 is obtained; then the judgment boundary of outliers is determined, that is, the values less than Q1-1.5xIQR or greater than Q3+1.5xIQR are determined as outliers; finally, these outliers are directly removed from the data set to avoid the interference of extreme data on subsequent processing; for the missing values in the attribute data, when the KNN algorithm is used to fill, first find the k most similar samples in the data set for each sample containing missing values, and the k value is pre-set according to the data distribution, then calculate the filling value of the missing value by weighted average weight according to the values of the k similar samples in the corresponding missing field, so as to complete the completion of the missing data and obtain the cleaned attribute data; When the category data such as brand name and product model in the cleaned attribute data is converted into a 32-dimensional vector by using the Word2Vec model, all category data is regarded as a text sequence, and the Skip-gram or CBOW algorithm of the Word2Vec model is used to train the text, so that the model learns the semantic association between different categories; then each category text is mapped into the trained 32-dimensional vector space, and each dimension of the vector reflects the characteristics of the category in a specific semantic dimension, realizing the numerical conversion of the category data; for the time series data such as maintenance record time and performance parameter monitoring time in the cleaned attribute data, the data is sorted according to the chronological order, so that the data can show a continuous change trend according to the time dimension; When the numerical data such as service life, original price and performance parameter value in the cleaned attribute data is standardized by Z-score, the mean and standard deviation of each numerical attribute are calculated, and then each data value in the attribute is converted into the form of original value-mean / standard deviation, so that the converted data has a mean of 0 and a standard deviation of 1; through this standardization processing, the influence of the differences in dimension and magnitude between different numerical attributes is eliminated, so that all numerical data is in the same order of magnitude, which is convenient for subsequent model to uniformly process and analyze different attribute characteristics, and finally the preprocessed attribute data is obtained.

[0022] Step 102, input the preprocessed image data and attribute data into the double-track multi-modal model, wherein the double-track multi-modal model includes an image processing track and a text processing track; In this embodiment, when the preprocessed image data and attribute data are input into the dual-track multimodal model, the preprocessed image data has completed segmentation, enhancement, denoising, size normalization and color correction and is sent to the image processing track. The track takes an improved convolutional neural network as the core, first extracts deep features of the image, captures visual information such as appearance, surface defects and wear degree of the commodity through multi-layer convolution and pooling operations; at the same time, the preprocessed attribute data is processed through cleaning, conversion and standardization, and the category vector, sorted time series data and standardized numerical values are input into the text processing track. The track first further optimizes the category features through the embedding layer, and then processes the time series data and numerical features by combining the hybrid encoder LSTM and the Transformer, to mine the correlation between attributes such as commodity brand, service life and maintenance record, so that the dual-track model synchronously analyzes the features of the commodity from the visual and text dimensions.

[0023] Step 103, the preprocessed image data enters the image processing track, and the image processing track extracts the visual feature vector of the second-hand commodity through the ResNet architecture; In this embodiment, the preprocessed image data is input into the ResNet architecture, and a plurality of convolution kernels are used in each convolution layer in the backbone network to extract local features of the image, and then the feature map size is reduced through the maximum pooling operation; the feature maps of corresponding scales are output at different levels of the backbone network and are sent to the feature pyramid network, and the 1 / 4, 1 / 8 and 1 / 16 scale feature maps are fused through top-down sampling and horizontal connection, to form a multi-scale fused feature map; the weight of each channel of the fused feature map is calculated through the channel attention module, and then the weight of each spatial position of the fused feature map is allocated through the spatial attention module, to automatically focus on the key regions of the second-hand commodity, wherein the key regions at least include the defect position and the severely worn region; the processed feature map continues to be feature-extracted through the convolution layer and the pooling layer, and finally is converted into a fixed-length vector through the global average pooling operation, to obtain the visual feature vector.

[0024] Step 104, the preprocessed attribute data enters the text processing track, and the text processing track encodes the feature of the commodity description text through the embedding layer and the hybrid encoder, to obtain the attribute feature vector; In this embodiment, the preprocessed attribute data is divided into category type attribute and numerical type attribute, the category type attribute is input into an embedding layer, and each category text is mapped into a 64-dimensional vector; the category type attribute vector output by the embedding layer is spliced with the numerical type attribute in field order to form an initial attribute feature matrix, wherein each row of the attribute feature matrix corresponds to an attribute feature combination; the initial attribute feature matrix is sorted in time order to form sequence data input into an LSTM module in the hybrid encoder, the LSTM module captures short-term time sequence dependency between attributes through a gating unit; the sequence data is input into a Transformer module in the hybrid encoder, the Transformer module calculates the association weight of each attribute with other attributes through a multi-head self-attention mechanism, captures long-distance feature dependency, and performs nonlinear conversion on the attention output through a feedforward neural network; the output features of the LSTM module and the Transformer module are fused by using a gating fusion mechanism to obtain the final attribute feature vector.

[0025] In step 105, the visual feature vector and the attribute feature vector are dynamically weighted and fused by using an attention fusion mechanism, the fused feature vector is input into a fully connected layer, and the grading result of the commodity is output.

[0026] In this embodiment, the visual feature vector output by the image processing track and the attribute feature vector output by the text processing track are obtained; the visual feature vector and the attribute feature vector are input into a feature interaction module in the attention fusion mechanism, the correlation score between the two vectors is calculated through dot product operation; the initial weight is set based on the commodity type classification result, and the initial weight is dynamically adjusted according to the correlation score; the visual feature vector and the attribute feature vector are weighted and summed according to the adjusted weight to obtain a fused comprehensive feature vector; the fused comprehensive feature vector is input into a fully connected layer, a softmax activation function is applied at the output end of the fully connected layer, the output value is converted into a probability distribution of each grade, and the grade with the highest probability is selected as the final grading result of the commodity.

[0027] Referring to Figure 2 The embodiment of the present application provides a deep learning-based double-track grading evaluation system for second-hand commodities, which comprises: A preprocessing module is configured to collect image data and attribute data of the second-hand commodities and perform preprocessing on the image data and the attribute data respectively to obtain preprocessed image data and attribute data. An input module is configured to input the preprocessed image data and attribute data into a double-track multi-modal model, wherein the double-track multi-modal model comprises an image processing track and a text processing track. An extraction module is configured to input the preprocessed image data into the image processing track, and the image processing track extracts a visual feature vector of the second-hand commodities through a ResNet architecture. The encoding module is used for the attribute data after preprocessing to enter a text processing track, and the text processing track is used for feature encoding on the commodity description text through an embedding layer and a hybrid encoder to obtain an attribute feature vector; The fusion module is used for dynamically weighting and fusing the visual feature vector and the attribute feature vector through an attention fusion mechanism, inputting the fused feature vector into a full connection layer, and outputting a grading result of the commodity.

[0028] Figure 3 The based deep learning's second-hand commodity double track grading evaluation equipment 300 can be different due to different configurations or performances, and can include one or more processors (central processing units, CPU) 310 (for example, one or more processors) and a memory 320, one or more storage media 330 (for example, one or more mass storage devices) storing application programs 333 or data 332. Wherein, the memory 320 and the storage medium 330 can be temporary storage or persistent storage. The program stored in the storage medium 330 can include one or more modules (not shown in the figure), each module can include a series of instruction operations in the based deep learning's second-hand commodity double track grading evaluation equipment 300. Further, the processor 310 can be arranged to communicate with the storage medium 330, execute a series of instruction operations in the storage medium 330 on the based deep learning's second-hand commodity double track grading evaluation equipment 300, so as to realize the method provided by the above embodiment.

[0029] The based deep learning's second-hand commodity double track grading evaluation equipment 300 can further include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 331, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that, Figure 3 The based deep learning's second-hand commodity double track grading evaluation equipment structure shown does not constitute a limitation on the computer equipment provided by the present application, and can include more or fewer components than the figure shown, or combine certain components, or different component arrangements.

[0030] The application further provides a computer readable storage medium, which can be a nonvolatile computer readable storage medium or a volatile computer readable storage medium, and instructions are stored in the computer readable storage medium, and the instructions make a computer execute steps of the deep learning based second-hand commodity double-track grading evaluation method provided by each embodiment when the instructions are run on the computer.

[0031] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device or apparatus, unit can refer to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0032] The integrated unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0033] The basic principles, main features and advantages of the application are shown and described above. Those skilled in the art should understand that the application is not limited by the above embodiments, and the above embodiments and the description in the specification are only preferred examples of the application and are not intended to limit the application. Without departing from the spirit and scope of the application, various changes and improvements can be made to the application, and these changes and improvements all fall within the scope of the claimed application. The scope of protection of the application is defined by the appended claims and their equivalents.

Claims

1. A deep learning-based dual-track grading evaluation method for second-hand goods, characterized by, The method comprises the following steps: Collecting image data and attribute data of second-hand commodities and respectively preprocessing to obtain preprocessed image data and attribute data; The preprocessed image data and attribute data are input into a double-track multi-modal model, wherein the double-track multi-modal model comprises an image processing track and a text processing track; The preprocessed image data enters the image processing track, and the image processing track extracts a visual feature vector of the second-hand commodity through a ResNet architecture; The preprocessed attribute data enters the text processing track, and the text processing track encodes the commodity description text through an embedding layer and a hybrid encoder to obtain an attribute feature vector; The visual feature vector and the attribute feature vector are dynamically weighted and fused through an attention fusion mechanism, and the fused feature vector is input into a fully connected layer to output a grading result of the commodity.

2. The deep learning-based dual-track grading evaluation method for second-hand goods according to claim 1, wherein, The collecting of the image data and attribute data of the second-hand commodities and the preprocessing to obtain the preprocessed image data and attribute data comprise: A U-Net model is used to extract a commodity main body region from the image data of the second-hand commodities to remove background interference; Data enhancement processing is performed through random rotation, horizontal flipping, brightness disturbance and Gaussian noise, and image denoising, size normalization and color correction processing are performed to obtain the preprocessed image data.

3. The deep learning-based dual-track grading evaluation method for second-hand goods according to claim 2, wherein, The U-Net model is used to extract the commodity main body region from the image data of the second-hand commodities to remove background interference, comprising: The second-hand commodity image to be processed is input into the U-Net model, local features of the image are extracted by using a plurality of convolution kernels, and then the local region maximum value is selected through a maximum pooling operation, the spatial size of the feature map is gradually reduced, and the number of feature channels is increased to obtain deep semantic features of the image; The second-hand commodity image to be processed and the deep semantic features are fused through a skip connection to form a fused feature map; The fused feature map is input into the output layer of the U-Net model, convolution operations are performed through the same convolution kernel as the commodity main body and the background, and the probability distribution of each pixel belonging to the commodity main body or the background is obtained through the softmax activation function; According to a preset probability threshold, the output probability distribution is judged to distinguish the background and the commodity main body, and the extraction of the commodity main body region is completed.

4. The deep learning-based dual-track grading evaluation method for second-hand goods according to claim 1, wherein, The collecting of the image data and attribute data of the second-hand commodities and the preprocessing to obtain the preprocessed image data and attribute data further comprise: An IQR method is used to detect and eliminate outliers of the attribute data of the second-hand commodities, and a KNN algorithm is used to fill in missing values in the attribute data to obtain cleaned attribute data; A Word2Vec model is used to convert the category type data in the cleaned attribute data into a 32-dimensional vector, and the time series data in the cleaned attribute data is sorted according to the time stamp; The numerical type data in the cleaned attribute data is subjected to Z-score standardization to obtain the preprocessed attribute data.

5. The dual-track grading evaluation method for used goods based on deep learning according to claim 1, wherein, The preprocessed image data enters the image processing track, and the image processing track extracts a visual feature vector of the second-hand commodity through a ResNet architecture, comprising: The preprocessed image data is input into a ResNet architecture, and multiple convolution kernels are used in each convolution layer in the backbone network to extract local features of the image, and then the feature map size is reduced through the maximum pooling operation; The feature maps of different scales are output at different levels of the backbone network and sent to the feature pyramid network, which fuses 1 / 4, 1 / 8, and 1 / 16 scale feature maps through top-down sampling and horizontal connection to form multi-scale fusion feature maps; First, the channel attention module is used to calculate the weight of each channel of the fusion feature map, and then the spatial attention module is used to assign weights to each spatial position of the fusion feature map, automatically focusing on the key regions of the second-hand goods, which at least include defect positions and severely worn areas; The processed feature map continues to be refined through convolution and pooling layers, and finally converted into a fixed-length vector through global average pooling operation to obtain a visual feature vector.

6. The dual-track grading evaluation method for used goods based on deep learning according to claim 1, wherein, The preprocessed attribute data enters the text processing track, and the text processing track encodes the feature of the commodity description text through the embedding layer and the hybrid encoder to obtain the attribute feature vector, including: The preprocessed attribute data is divided into categorical attributes and numerical attributes, and the categorical attributes are input into the embedding layer to map each category text into a 64-dimensional vector; The categorical attribute vector output by the embedding layer is concatenated with the numerical attributes in field order to form an initial attribute feature matrix, where each row of the attribute feature matrix corresponds to an attribute feature combination; The initial attribute feature matrix is sorted in time order to form sequence data input into the LSTM module in the hybrid encoder, which captures the short-term temporal dependence between attributes through the gating unit; The sequence data is input into the Transformer module in the hybrid encoder, which calculates the association weight between each attribute and other attributes through the multi-head self-attention mechanism to capture long-distance feature dependence, and performs nonlinear transformation on the attention output through the feedforward neural network; The output features of the LSTM module and the Transformer module are fused using the gating fusion mechanism to obtain the final attribute feature vector.

7. The dual-track grading evaluation method for used goods based on deep learning according to claim 1, wherein, The visual feature vector and the attribute feature vector are dynamically weighted and fused through the attention fusion mechanism, and the fused feature vector is input into the fully connected layer to output the grading result of the commodity, including: Obtain the visual feature vector output by the image processing track and the attribute feature vector output by the text processing track; The visual feature vector and the attribute feature vector are input into the feature interaction module in the attention fusion mechanism, and the correlation score between the two vectors is calculated through the dot product operation; Based on the commodity type classification result, set the initial weight and dynamically adjust the initial weight according to the correlation score; The visual feature vector and the attribute feature vector are weighted and summed according to the adjusted weight to obtain the fused comprehensive feature vector; The integrated feature vector after fusion is input into a full connection layer, a softmax activation function is applied at the output end of the full connection layer to convert the output value into a probability distribution of each grade, and the grade with the highest probability is selected as the final grading result of the commodity.

8. A deep learning-based two-track grading evaluation system for second-hand goods, characterized by, The system comprises: A preprocessing module is configured to collect image data and attribute data of the second-hand commodity and perform preprocessing on the image data and the attribute data respectively to obtain preprocessed image data and attribute data. An input module is configured to input the preprocessed image data and attribute data into a double-track multi-modal model, wherein the double-track multi-modal model comprises an image processing track and a text processing track. An extraction module is configured to input the preprocessed image data into the image processing track, and the image processing track extracts a visual feature vector of the second-hand commodity through a ResNet architecture. An encoding module is configured to input the preprocessed attribute data into the text processing track, and the text processing track encodes a feature of a commodity description text through an embedding layer and a hybrid encoder to obtain an attribute feature vector. A fusion module is configured to perform dynamic weighted fusion on the visual feature vector and the attribute feature vector through an attention fusion mechanism, input the fused feature vector into a full connection layer, and output a grading result of the commodity.

9. A deep learning-based two-track grading evaluation device for second-hand goods, characterized by, The deep learning-based double-track grading evaluation device for second-hand commodities comprises a memory and at least one processor, and the memory stores instructions; the at least one processor invokes the instructions in the memory to enable the deep learning-based double-track grading evaluation device for second-hand commodities to perform the steps of the deep learning-based double-track grading evaluation method for second-hand commodities according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon instructions, the instructions comprising, The instructions are executed by the processor to implement the steps of the deep learning-based double-track grading evaluation method for second-hand commodities according to any one of claims 1-7.

Images