A computer vision assisted logistics transfer field digital twin assisted modeling method

Abstract

The application discloses a computer vision assisted logistics transfer field digital twin assisted modeling method, comprising the following steps: S1, collecting visual data of key areas of the logistics transfer field, and carrying out noise suppression and quality enhancement on the collected visual data; S2, extracting features of the image after noise suppression and quality enhancement by training a YOLO network to obtain visual features; S3, collecting and encoding non-visual data to obtain non-visual features; S4, performing image segmentation, scene recognition and object tracking on the preprocessed image by using a deep learning method; and S5, performing multi-modal data fusion and digital twin model construction.The application realizes global dynamic monitoring and high-precision digital twin modeling of key areas of the logistics transfer field.

Description

Technical Field

[0001] This invention relates to the field of logistics, and in particular to a computer vision-assisted digital twin modeling method for logistics transit sites. Background Technology

[0002] Current logistics transit points (such as sorting lines, warehouse aisles, loading and unloading areas, etc.) experience rapid changes in dynamic information due to high material flow and complex operating environments. Traditional monitoring often relies on single sensor data or low-resolution video, making it difficult to comprehensively reflect operational status, cargo flow, and personnel location. Although some technologies have attempted to fuse RFID, environmental sensor, and camera data, the following shortcomings remain in data preprocessing, semantic extraction, and multi-source information fusion: 1. Insufficient visual data processing capabilities: Existing systems have limited capabilities in video and image noise reduction and image enhancement, failing to fully utilize the detailed information captured by high-resolution cameras. 2. Inaccurate target detection and tracking: Current target detection and image segmentation algorithms suffer from false detections or missed detections in complex scenarios, making it difficult to accurately obtain cargo flow and personnel location information. 3. Lack of multimodal data fusion: The spatiotemporal dynamic correlation analysis between visual information and RFID and environmental data is not in-depth enough, resulting in significant deviations in the digital twin model and an inability to fully recreate the physical structure and dynamic state of the site. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a computer vision-assisted digital twin modeling method. Through multi-source data acquisition, edge preprocessing, and deep learning technology, it can achieve global dynamic monitoring and high-precision digital twin modeling of key areas in logistics transit.

[0004] The objective of this invention is achieved through the following technical solution: a computer vision-assisted digital twin modeling method, comprising the following steps:

[0005] S1. Collect visual data of key areas in the logistics transfer center, and perform noise suppression and quality enhancement on the collected visual data;

[0006] S2. Use the trained YOLO network to extract features from the noise-suppressed and quality-enhanced images to provide visual features for subsequent object detection and multimodal fusion;

[0007] S3. Perform non-visual data acquisition and encoding to obtain non-visual features;

[0008] S4. Image after preprocessing The above utilizes deep learning methods for image segmentation, scene recognition, and object tracking.

[0009] S5. Perform multimodal data fusion and digital twin model construction.

[0010] The beneficial effects of this invention are as follows: This invention constructs a system for digital twin-assisted modeling of logistics transit centers. By deploying high-resolution cameras, RFID, temperature and humidity sensors, gas sensors, and other key monitoring equipment in key areas such as sorting lines, warehouse aisles, and loading / unloading areas, it achieves real-time synchronous acquisition and preprocessing of on-site video, image, and environmental data. The system adopts an advanced edge computing platform and utilizes image enhancement and denoising technology based on generative adversarial networks (GANs) to restore noisy images with high quality, ensuring the input quality of subsequent visual processing modules. On this basis, a pre-trained YOLO network performs target detection and feature extraction on the enhanced images, effectively identifying the boundaries, positions, and category information of each target in the image. Simultaneously, image segmentation technology is used to achieve pixel-level semantic segmentation, further obtaining fine-grained scene information. By vectorizing and deeply fusing target detection results, image segmentation results, scene recognition information, and non-visual sensor data, the system establishes a comprehensive descriptive vector reflecting the global state of the logistics transit center, thereby realizing real-time mapping and bidirectional data synchronization between the on-site physical structure and the digital twin model. This system not only significantly improves the ability to perceive on-site operational status, personnel and cargo flow, and environmental changes, but also provides accurate and comprehensive data support for subsequent fault prediction, maintenance early warning, and intelligent scheduling. It overcomes the shortcomings of traditional periodic maintenance modes in real-time monitoring and early warning, and provides new theoretical basis and technical support for intelligent logistics equipment management and maintenance. Attached Figure Description

[0011] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0012] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the following description.

[0013] like Figure 1 As shown, a computer vision-assisted digital twin modeling method includes the following steps:

[0014] S1. Collect visual data of key areas in the logistics transfer center, and perform noise suppression and quality enhancement on the collected visual data;

[0015] Visual data acquisition and preprocessing:

[0016] High-resolution cameras are deployed in key areas of the logistics transit center (such as sorting lines, warehouse aisles, and loading / unloading areas) to collect video and image data in real time. Let the original image frame be represented as:

[0017] ;

[0018] in, Indicates the image height. Indicates width, Here is the number of color channels. The image undergoes noise suppression and quality enhancement using an edge computing device; the enhanced image is defined as follows:

[0019] ;

[0020] in, This describes an image enhancement and denoising algorithm based on Generative Adversarial Networks (GANs). The main architecture is as follows: The generator (G) uses an encoder-decoder (or U-Net structure) to recover a high-quality image from a noisy image; the discriminator (D) is used to distinguish the difference between the generated image and the high-quality real image.

[0021] Image input: The input noisy image and the input high-quality real image are denoted as:

[0022] ;

[0023] Generator The generator produces noisy images. The input is the enhanced image, and the output is the image itself.

[0024] ;

[0025] The generator typically consists of convolutional layers, downsampling layers (encoder part), corresponding upsampling layers (decoder part), and skip connections to fully utilize local and global features to recover image details.

[0026] Discriminator The discriminator takes an input image (a real image or an output image from the generator) and determines its realism, i.e., it outputs a probability.

[0027] ;

[0028] Loss function: The global objective of GAN consists of two main parts, including adversarial loss and content loss.

[0029] Adversarial loss: This allows the image generated by the generator to "fool" the discriminator. The following losses apply to both the generator and the discriminator:

[0030] ;

[0031] Content loss: To ensure that the generated image is as consistent as possible with the real image in terms of content, L1 loss (or L2 loss) is introduced:

[0032] ;

[0033] Global generator objective: The generator's objective function combines adversarial loss and content loss.

[0034] ;

[0035] in, This is a weighting factor.

[0036] In the actual training process, multiple batches of samples are collected. Each sample contains several sample groups, and each sample group consists of noisy images and corresponding real high-quality images.

[0037] For each batch of samples, the global generator objective is obtained as the loss function, and then the Generative Adversarial Network (GAN) is optimized using mini-batch gradient descent until the GAN converges and all batches of samples are trained, resulting in a trained GAN. In actual processing, the image to be processed is fed into the generator, thus obtaining the image after noise suppression and quality enhancement.

[0038] In the embodiments of this application, optimization is achieved through backpropagation and gradient descent (Adam). The overall training objective of the GAN is as follows: ;

[0039] In actual training, G is fixed and D is optimized (maximizing the discriminator accuracy), and D is fixed and G is optimized (minimizing the difference between the generated image and the real image), and this process is repeated until convergence.

[0040] Training requires multiple sets of samples, each set including noisy images. and the corresponding real image .

[0041] S2. Use the trained YOLO network to extract features from the noise-suppressed and quality-enhanced images, and provide high-level semantic information (visual features) for subsequent object detection and multimodal fusion;

[0042] Step S2 includes:

[0043] The YOLO network is used to perform object detection and feature extraction on the image processed in step S1, and the output feature map is as follows:

[0044] ;

[0045] in, The number of grids into which the input image is divided; This represents the number of bounding boxes predicted for each grid cell; 5 indicates the 5-dimensionality of the five parameters, which are: bounding box center coordinates... Width and Height and confidence level ; Number of categories; These are the network parameters; the information output by the YOLO network also includes whether a target was detected in each bounding box.

[0046] For the The first grid Each bounding box contains predicted values. in It is a class probability vector that satisfies ; For the first The grid and the first The confidence score, center x-coordinate, center y-coordinate, width, and height of each bounding box prediction; express The c-th element in;

[0047] During the training process, the YOLO network needs to acquire multiple sets of samples. Each set of samples includes an input image and a desired feature map of the input image. The input image is input into the YOLO network, and the loss function is calculated by the feature map output by the YOLO network and the desired feature map. Then, based on the loss function, the YOLO network is updated using the gradient descent method until convergence or all samples are trained, resulting in a trained YOLO network. Convergence means that the loss function value is less than a preset threshold.

[0048] The expected feature map also includes, Number of grid cells; Number of bounding boxes predicted for each grid cell is... , No. The first grid A bounding box, containing parameters including , respectively representing the first The grid and the first The expected confidence score, center x-coordinate, center y-coordinate, width, height, and class probability vector of each bounding box; ; express The c-th element in;

[0049] The input image is processed using a trained YOLO network to obtain a visual feature vector; that is, the output of the YOLO network is taken as the visual feature vector for subsequent multimodal data fusion.

[0050] ;

[0051] The loss function of the YOLO network consists of three components: coordinate loss, confidence loss, and classification loss.

[0052] The coordinate loss is defined as follows:

[0053] ;

[0054] in, The weights represent the coordinate loss; if the first... The grid and the first The target was detected in one bounding box. =1, otherwise =0;

[0055] At the same time, the width and height are processed by taking the square root:

[0056] ;

[0057] Confidence loss is defined as:

[0058] ;

[0059] Add confidence loss to bounding boxes where no target was detected:

[0060] in, This indicates the confidence weight when no target is detected;

[0061] The classification loss is defined as:

[0062] ;

[0063] Overall loss synthesis:

[0064] ;

[0065] S3. Perform non-visual data acquisition and encoding to obtain non-visual features;

[0066] Multi-source sensor data acquisition: Simultaneously deploy multiple sensors for data acquisition; set time... The non-visual data vector is:

[0067] ;

[0068] in Indicates the number of sensors. This represents the information collected by the i-th sensor; the sensors include RFID, temperature and humidity, and gas environment sensors.

[0069] Time synchronization mechanism: Due to the high frame rate of visual data and the low sampling frequency of non-visual data, non-visual data is synchronized within a certain time window. After averaging or interpolating, the aligned data is obtained.

[0070] ;

[0071] Data normalization and feature encoding: Data from different sensors is first normalized, then encoded and mapped to a high-dimensional feature space through a fully connected layer to obtain non-visual feature representations.

[0072] ;

[0073] in This is the weight matrix. For bias, For activation function, Indicates the dimension of the encoded features;

[0074] Obtain visual features respectively and encoded non-visual features Subsequently, feature-level fusion will be performed to form a unified multimodal description.

[0075] S4. Image after preprocessing The above utilizes deep learning methods for image segmentation, scene recognition, and object tracking.

[0076] S401. Image Segmentation and Scene Recognition:

[0077] Image segmentation: Using the Mask R-CNN network to segment images Perform pixel-level classification and output a segmentation mask:

[0078] ;

[0079] in, The function representing the Mask R-CNN network, This represents the parameters of the Mask R-CNN network;

[0080] During the training of the Mask R-CNN network, a sample set consisting of multiple samples needs to be constructed to train the model; each sample contains the image to be segmented and the desired segmentation mask; after training, it can be directly used for image segmentation. The division;

[0081] Scene recognition: using scene classification networks (such as CNN networks) to classify images Mapped to scene category probability vector :

[0082] ;

[0083] in, A function for scene classification networks. Parameters for the scene classification network;

[0084] When training a scene classification network, a sample set consisting of multiple samples needs to be constructed to train the model; each sample contains the input image and the desired class probability vector; after training, it can be directly used for image processing. Mapping;

[0085] in The total number of scene categories, satisfying ;

[0086] S402. Object Tracking:

[0087] In consecutive frames, through adjacent frames With frames The matching cost is calculated based on the spatial and appearance features of the target. The cost function is defined as follows:

[0088] ;

[0089] Among them, for frames No. The appearance feature vector of each detection box The feature vector contains the bounding box parameters of the i-th detection box. The feature vector corresponding to the i-th detection box The Euclidean distance between the target locations; Distance to the target's appearance features The adjustment coefficient is calculated based on the defined cost function. and The matching cost between them is then used to construct the cost matrix:

[0090] ;

[0091] in,

[0092] ;

[0093] The constructed cost matrix is ​​transformed into a binary assignment problem using the Hungarian algorithm, and the optimal one-to-one match is obtained in polynomial time through the following steps:

[0094] (1) Perform row subtraction for each row: This ensures that each row contains at least one 0.

[0095] (2) Perform column subtraction on each column: This ensures that each column contains at least one 0.

[0096] (3) Find the fewest horizontal / vertical lines to cover all zero elements; if the required number of lines is less than Then let And perform the following: For each uncovered element, update the value. For elements covered by only one line, keep the value unchanged; for elements covered by the intersection of two lines, update the value. Repeat until the number of covered lines equals 1. .

[0097] (4) Extract one-to-one matches from the covered zero elements, and let the corresponding To minimize . This is a binary variable representing a one-to-one matching situation, i.e.

[0098] The algorithm will output a one-to-one pairing scheme, allowing the frames to... Each detection box in the image corresponds exactly to a frame. A detection box in the frame enables target association and continuous tracking across consecutive frames.

[0099] S5. Perform multimodal data fusion and digital twin model construction.

[0100] (1) Multimodal feature fusion:

[0101] Feature-level fusion: combining visual features With non-visual coding features By concatenating the vectors, a unified fusion feature vector is formed.

[0102] ;

[0103] in, This indicates a vector concatenation operation.

[0104] Attention Mechanism Weighting: This method utilizes an attention network to adaptively adjust the contributions of information from different modalities by calculating attention weights.

[0105] ;

[0106] The final weighted fusion feature is obtained:

[0107] ;

[0108] in, These are the parameters for the attention network.

[0109] Temporal modeling: Due to the temporal continuity of the logistics site's state, an LSTM model is used to capture temporal dependencies and update the state representation.

[0110] ;

[0111] During the training of the LSTM model, multiple sets of samples need to be collected for training. The samples in each set are obtained at different values ​​of t. The features of each set of samples are the weighted fusion features at time t. The logistics site status at time t-1 The status of the logistics site at time t is labeled. ;

[0112] After training, for any time t, it is only necessary to combine the weighted fused features at time t. The logistics site status at time t-1 Input an LSTM model, and the LSTM model will output the logistics site status at time t. The prediction results;

[0113] (2) Digital twin model construction and dynamic updating:

[0114] Constructing a state description vector: Based on object detection, image segmentation, scene recognition, and non-visual data, a composite description of the state of the logistics transit site is formed.

[0115] ;

[0116] in, , For deep fusion network mapping functions, To vectorize the detection and segmentation results.

[0117] Dynamic state update: Utilizing an exponential smoothing update strategy to achieve a smooth transition of state information.

[0118] ;

[0119] in, This is a smoothing factor. To further improve prediction accuracy, a Kalman filter is introduced for state correction:

[0120] ;

[0121] ;

[0122] in, Here is the state transition matrix. To control the input matrix, For external control input, This is the Kalman gain.

[0123] based on , By combining the image segmentation results and scene recognition results from step S402 with the target tracking scheme as an aid, a high-precision dynamic digital twin model of the logistics transfer center can be constructed.

[0124] In the embodiments of this application, the mapping between physical and virtual space can consider a coordinate mapping function: to achieve the mapping between the physical space on site and the coordinates of the digital twin model, an affine transformation function is introduced:

[0125] ;

[0126] in, For rotation and scaling matrices, It is a translation vector. These are the transformation parameters.

[0127] Minimize mapping error: Optimize mapping parameters using the least squares method and construct the objective function:

[0128] ;

[0129] In summary, this invention features real-time acquisition and preprocessing of multi-source data: High-resolution cameras are deployed in key areas of the logistics transit center, combined with RFID and environmental sensors, to achieve synchronous and real-time acquisition of on-site video, images, and non-visual data, followed by effective noise reduction and image enhancement preprocessing. Deep semantic extraction of visual data is also performed: Deep learning technology is used to perform target detection, scene recognition, object tracking, and image segmentation on the preprocessed visual data to extract key information such as operational status, cargo flow, and personnel location, ensuring high accuracy and robustness in complex scenarios. Finally, multimodal data fusion and digital twin modeling are implemented: The computer vision processing results are fused in real-time with other data such as RFID and environmental sensors to construct a high-precision digital twin model of the logistics transit center, comprehensively restoring the on-site physical structure and dynamic operating status, achieving virtual mapping and real-time feedback.

[0130] The above description represents preferred embodiments of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technical or knowledge. Any modifications and alterations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A computer vision-assisted digital twin-based modeling method for logistics transit sites, characterized in that: Includes the following steps: S1. Collect visual data of key areas in the logistics transfer center, and perform noise suppression and quality enhancement on the collected visual data; S2. Visual features are obtained by training a YOLO network to extract features from the noise-suppressed and quality-enhanced images; S3. Perform non-visual data acquisition and encoding to obtain non-visual features; S4. Image after preprocessing The above utilizes deep learning methods for image segmentation, scene recognition, and object tracking; Step S4 includes: S401. Image Segmentation and Scene Recognition: Image segmentation: Using the Mask R-CNN network to segment images Perform pixel-level classification and output a segmentation mask: ； in, The function representing the Mask R-CNN network, This represents the parameters of the Mask R-CNN network; Indicates the image height. Indicates width; Scene recognition: using scene classification networks to classify images Mapped to scene category probability vector : ； in, A function for scene classification networks. Parameters for the scene classification network; in The total number of scene categories, satisfying ; S402. Object Tracking: In consecutive frames, through adjacent frames With frames The matching cost is calculated based on the spatial and appearance features of the target. The cost function is defined as follows: ； Among them, for frames No. The appearance feature vector of each detection box The feature vector contains the bounding box parameters of the i-th detection box. The feature vector corresponding to the i-th detection box The Euclidean distance between the target locations; Distance to the target's appearance features The adjustment coefficient is calculated based on the defined cost function. and The matching cost between them is then used to construct the cost matrix: ； in, ； The constructed cost matrix is ​​transformed into a binary assignment problem using the Hungarian algorithm. The optimal one-to-one matching is then obtained in polynomial time through the following steps. The Hungarian algorithm outputs a one-to-one pairing scheme as the target tracking scheme, allowing the frame... Each detection box in the image corresponds exactly to a frame. A detection box in one frame enables target association and continuous tracking across consecutive frames; S5. Perform multimodal data fusion and construct digital twin models; Step S5 includes: S501. Multimodal Feature Fusion: Feature-level fusion: combining visual features With non-visual coding features By concatenating the vectors, a unified fused feature vector is formed: ； in, Indicates a vector concatenation operation; Attention Mechanism Weighting: This method utilizes an attention network to adaptively adjust the contributions of information from different modalities by calculating attention weights. ； The final weighted fusion feature is obtained: ； in, These are the parameters of the attention network; This indicates element-wise multiplication; Temporal modeling: Due to the temporal continuity of the logistics site's state, an LSTM model is used to capture temporal dependencies and update the state representation. ； The logistics site status includes: the real-time location status of packages in the logistics transfer center, the sorting status of the sorting machine, and the location status of sorting personnel. During the training of the LSTM model, multiple sets of samples need to be collected for training. The samples in each set are obtained at different values ​​of t. The features of each set of samples are the weighted fusion features at time t. The logistics site status at time t-1 The status of the logistics site at time t is labeled. ; After training, for any time t, it is only necessary to combine the weighted fused features at time t. The logistics site status at time t-1 Input an LSTM model, and the LSTM model will output the logistics site status at time t. The prediction results; S502. Digital Twin Model Construction and Dynamic Updates: Constructing a state description vector: Based on object detection, image segmentation, scene recognition, and non-visual data, a composite description of the state of the logistics transit site is formed. ； in, , For deep fusion network mapping functions, To vectorize the detection and segmentation results; Indicates time Non-visual data vectors; Dynamic state update: Utilizing an exponential smoothing update strategy to achieve a smooth transition of state information. ； in, It is a smoothing factor; Introducing Kalman filtering for state correction: ； ； in, Here is the state transition matrix. To control the input matrix, For external control input, For the Kalman gain; for the state transition matrix This refers to the real-time location status of packages, the sorting status of sorting machines, and the location status changes of sorting personnel from the current moment to the next moment, based on the physical model of the sorting system; external control inputs. This refers to the input decision information for logistics transit centers, including the allocation of parcel sorting slots, conveyor belt speeds, and the scheduling of sorting personnel; the control input matrix. That is, external control input The following factors affect the operational status of the logistics sorting system; S503. Based on , By combining the image segmentation results and scene recognition results from step S402 with the target tracking scheme as an aid, a high-precision dynamic digital twin model of the logistics transfer center can be constructed.

2. The computer vision-assisted digital twin modeling method for logistics transit centers according to claim 1, characterized in that: The key areas include the sorting line, warehouse aisles, and loading / unloading area of ​​the logistics transit center; high-resolution cameras are deployed in the key areas to collect video image data in real time, which serves as visual data for the key areas.

3. The computer vision-assisted digital twin modeling method for logistics transit centers according to claim 2, characterized in that: Step S1 includes: Let the original image frame be represented as: ； in, Indicates the image height. Indicates width, The image is represented by the number of color channels. Noise suppression and quality enhancement are performed on the image using an edge computing device. The enhanced image is defined as follows: ； in, This represents an image enhancement and denoising algorithm based on generative adversarial networks.

4. The computer vision-assisted digital twin modeling method for logistics transit centers according to claim 3, characterized in that: The architecture of the image enhancement and denoising algorithm based on Generative Adversarial Network (GAN) includes: a generator for recovering a high-quality image from a noisy image; and a discriminator for identifying the difference between the generator's output image and the real image. The model training process for the image enhancement and denoising algorithm based on Generative Adversarial Networks (GANs) is as follows: A1. Image Input: The input noisy image and the real image are denoted as follows: ； A2, Generator The generator produces noisy images. The input is the enhanced image, and the output is the image itself. ； The generator consists of convolutional layers, downsampling layers, upsampling layers, and skip connections; the convolutional layers and downsampling layers form the encoder part, while the upsampling layers and skip connections form the decoder part. A3, Discriminator The discriminator judges the authenticity of a real image or the generator's output image by taking it as input, i.e., it outputs a probability. : ； The input to the discriminator can be a real image or the output image of the generator; the discriminator is composed of convolutional layers, pooling layers, and fully connected layers. A4. Loss Function: The global objective of GAN consists of two parts, including adversarial loss and content loss, both of which are loss functions designed for a batch of samples. Adversarial Loss: The generator and discriminator have the following adversarial loss: ； in, Used to optimize the discriminator D, with the aim of enabling D to recognize real images. This represents the discriminative expectation of the current batch of real images, i.e., for multiple images in the current batch... corresponding Calculate the average value; Used to optimize the generator G, with the aim of making G "fool" the discriminator, making the generated images look more realistic. Let's calculate the expected value of the discrimination of the current batch of real high-quality images, i.e.: For multiple in the current batch corresponding Calculate the average value; Content loss: To ensure that the generated image is as consistent as possible with the real image in terms of content, L1 loss is introduced. The content loss is denoted as: ； express L1 norm; This represents the expected content loss, i.e., the loss of the current batch of samples. Calculate the average value; Global generator objective: The generator's objective function combines adversarial loss and content loss. ； in, For the weighting factor; A5. In actual training, the training should be conducted in the following manner: In the actual training process, multiple batches of samples are collected. Each sample contains several sample groups, and each sample group consists of noisy images and corresponding real high-quality images. For each batch of samples, the global generator objective is obtained as the loss function according to steps A1~A4. Then, the generative adversarial network (GAN) is optimized by mini-batch gradient descent until the GAN converges and all batches of samples are trained. The trained GAN is then obtained. In the actual processing, the image to be processed is fed into the generator to obtain the image after noise suppression and quality enhancement.

5. The computer vision-assisted digital twin modeling method for logistics transit centers according to claim 1, characterized in that: Step S2 includes: The YOLO network is used to perform object detection and feature extraction on the image processed in step S1, and the output feature map is as follows: ； in This represents the processing function of the YOLO network. The number of grids into which the input image is divided; This represents the number of bounding boxes predicted for each grid cell; 5 indicates the 5-dimensionality of the five parameters, which are: bounding box center coordinates... Width and Height and confidence level ; Number of categories; These are the network parameters; the information output by the YOLO network also includes whether a target was detected in each bounding box. For the The first grid Each bounding box contains predicted values. in It is a class probability vector that satisfies ; For the first The grid and the first The confidence score, center x-coordinate, center y-coordinate, width, and height of each bounding box prediction; express The c-th element in; During the training process, the YOLO network needs to acquire multiple sets of samples. Each set of samples includes an input image and a desired feature map of the input image. The input image is input into the YOLO network, and the loss function is calculated by the feature map output by the YOLO network and the desired feature map. Then, based on the loss function, the YOLO network is updated using the gradient descent method until convergence or all samples are trained, resulting in a trained YOLO network. Convergence means that the loss function value is less than a preset threshold. The expected feature map also includes, Number of grid cells; Number of bounding boxes predicted for each grid cell is... , No. The first grid A bounding box, containing parameters including , respectively representing the first The grid and the first The expected confidence score, center x-coordinate, center y-coordinate, width, height, and class probability vector of each bounding box; ; express The c-th element in; The input image is processed using a trained YOLO network to obtain a visual feature vector; that is, the output of the YOLO network is taken as the visual feature vector. ； The loss function of the YOLO network consists of three components: coordinate loss, confidence loss, and classification loss. The coordinate loss is defined as follows: ； in, The weights represent the coordinate loss; if the first... The grid and the first The target was detected in one bounding box. =1, otherwise =0; At the same time, the width and height are processed by taking the square root: ； Confidence loss is defined as: ； Add confidence loss to bounding boxes where no target was detected: in, This indicates the confidence weight when no target is detected; The classification loss is defined as: ； Overall loss synthesis: 。 6. The computer vision-assisted digital twin modeling method for logistics transit centers according to claim 5, characterized in that: Step S3 includes: Multi-source sensor data acquisition: Simultaneously deploy multiple sensors for data acquisition; set time... The non-visual data vector is: ； in Indicates the number of sensors. This represents the information collected by the i-th sensor; the sensors include RFID, temperature and humidity, and gas environment sensors. Time synchronization mechanism: Due to the high frame rate of visual data and the low sampling frequency of non-visual data, non-visual data is synchronized within a certain time window. After averaging or interpolating, the aligned data is obtained. ； Data normalization and feature encoding: Data from different sensors is first normalized, then encoded and mapped to a high-dimensional feature space through a fully connected layer to obtain non-visual feature representations. ； in This is the weight matrix. For bias, For activation function, Indicates the dimension of the encoded features; Obtain visual features respectively and encoded non-visual features Subsequently, feature-level fusion will be performed to form a unified multimodal description.

Images