Intelligent port timber counting method and device based on depth estimation

Abstract

The embodiment of the invention discloses an intelligent port wood counting method and device based on depth estimation. The method comprises the steps of obtaining an original image of a target scene in response to a capture operation completion instruction, determining a depth map corresponding to the original image based on a monocular depth estimation model, and splicing the original image and the depth map into four-channel data; based on the four-channel data, the trained target detection model outputs an identification result, the number of input channels of a first layer of convolution of the target detection model is four, and all the channels are read when the data are loaded; and determining the average depth of the grabbed targets in the grabbing operation based on the depth values of the grabbed targets, and counting the remaining grabbed targets after removing the grabbed targets of which the deviation proportions with the average depth are greater than a proportion threshold. In the embodiment of the invention, the interference objects not in the grabbing structure can be better eliminated, so that the number of the target objects in the grabbing structure can be more accurately identified and counted, and the automation level in the scene is improved.

Description

Technical Field

[0001] The embodiments in this specification belong to the field of image object recognition, and in particular relate to a method and apparatus for intelligent counting of port timber based on depth estimation. Background Technology

[0002] In port timber tallying operations, accurately counting the amount of timber in the grab bucket of gantry cranes or forklifts is crucial for improving operational efficiency and reducing human error. However, existing deep learning-based object detection methods (such as YOLO11) struggle to accurately distinguish the timber inside the grab bucket from the surrounding stacked timber in the complex environment of port terminals. This is because other timber is present in front of, behind, to the sides of the grab bucket, and the timber inside the grab bucket has similar features to the surrounding stacked timber in RGB images. Consequently, the counting accuracy of the timber inside the grab bucket is low, affecting operational efficiency. Summary of the Invention

[0003] The embodiments of this disclosure provide a method and apparatus for intelligent counting of port timber based on depth estimation, which aims to solve one or more of the above-mentioned problems and other potential problems.

[0004] According to a first aspect of this disclosure, a method for intelligent counting of port timber based on depth estimation is provided. The method includes: in response to a grasping operation completion instruction, acquiring an original image of the target scene; determining a depth map corresponding to the original image based on a monocular depth estimation model; stitching the original image and the depth map together to form four-channel data; based on the four-channel data, outputting a recognition result from a trained target detection model, wherein the first convolutional layer of the target detection model has four input channels and all channels are read during data loading; the recognition result is marked with the target box corresponding to the grasping structure and the grasped target within the target box; determining the average depth of the grasped target in this grasping operation based on the depth value of each grasped target; removing grasped targets whose deviation from the average depth is greater than a threshold; and counting the remaining grasped targets.

[0005] According to a second aspect of this disclosure, a port timber intelligent counting device based on depth estimation is provided. The device includes an image stitching module configured to acquire an original image of the target scene in response to a grasping operation completion command, and determine a depth map corresponding to the original image based on a monocular depth estimation model, so as to stitch the original image and the depth map into four-channel data; a model prediction module configured to output a recognition result from a trained target detection model based on the four-channel data, wherein the first convolutional layer of the target detection model has four input channels, and all channels are read when the data is loaded, and the recognition result is marked with the target box corresponding to the grasping structure and the grasped target within the target box; and a target counting module configured to determine the average depth of the grasped target in this grasping operation based on the depth value of each grasped target, and remove grasped targets whose deviation from the average depth is greater than a threshold, and count the remaining grasped targets.

[0006] According to a third aspect of this disclosure, an electronic device is provided, including one or more processors and a memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform a method provided according to a first scheme.

[0007] According to a fourth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided according to the first aspect.

[0008] The method provided in this specification can enhance the spatial positioning capability of the target detection model by introducing depth information obtained from a monocular depth estimation model and embedding the depth information as a fourth band into the target detection model. This can better eliminate interference objects that are not inside the grasping structure, thereby more accurately identifying and counting the number of target objects inside the grasping structure, and improving the automation level and work efficiency in this scenario. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein: Figure 1 A flowchart illustrating a depth-estimation-based intelligent counting method for port timber, representing some embodiments of this disclosure, is shown. Figure 2 A schematic diagram illustrating the principle process of the target detection model in the training phase of some embodiments of this disclosure is shown; Figure 3 A schematic diagram of the structure of a port timber intelligent counting device based on depth estimation, according to some embodiments of the present disclosure, is shown. Figure 4A schematic block diagram of an electronic device according to some embodiments of the present disclosure is shown. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0011] The terms “comprising” and “having”, and any variations thereof, in this specification, claims, and the foregoing drawings are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. Depending on the context, the word “if” as it applies herein may be interpreted as “when”, “in response to determination”, or “in response to detection”.

[0012] Figure 1 A flowchart illustrating a depth-estimation-based intelligent counting method 100 for port timber according to some embodiments of this disclosure is shown. Method 100 can be executed, for example, by a terminal, which may include, but is not limited to, mobile phones, tablets, desktop computers, servers, etc. Figure 1 As shown in box 102, method 100 can respond to the capture operation completion instruction, acquire the original image of the target scene, determine the depth map corresponding to the original image based on the monocular depth estimation model, and stitch the original image and the depth map into four-channel data.

[0013] In this embodiment, the target scene can be a complex environment such as a port or dock operation, where a grabbing structure capable of grasping the target exists. This grabbing structure can be a gantry crane grab, a forklift grab, etc., and the target can be a cylindrical material with a cross-section, such as wood or steel. After completing a grabbing operation, the grabbing structure can send a grabbing operation completion command to the terminal, informing the terminal that the grabbing is complete and image acquisition can begin. Upon receiving the grabbing operation completion command, the terminal can acquire images of the target scene using a camera positioned in the target scene to obtain the original image. To facilitate better subsequent recognition, the grabbing posture of the grabbing structure can be specified, or the camera can be positioned at a specific angle, so that the camera captures images corresponding to the cross-section of the target, facilitating counting based on the number of cross-sections.

[0014] Furthermore, the acquired raw images can be processed using a monocular depth estimation model to obtain a depth map. For example, Depth Anything V2 can be used as a monocular depth estimation model. Depth Anything V2 is a deep learning-based monocular depth estimation model that employs large-scale unsupervised pre-training and an efficient feature extraction architecture, capable of predicting scene depth information from a single RGB image. This model typically uses multi-scale feature fusion and attention mechanisms to improve the accuracy of depth estimation, making it suitable for 3D perception tasks in complex scenes. Since the monocular depth estimation model used in this application has not been modified, existing pre-trained models can be used directly, or the model can be trained according to the conventional model training process.

[0015] After obtaining the depth map corresponding to the original image, the original image and the depth map can be stitched together to obtain four-channel data. Specifically, the stitching method can be to normalize the depth map to 0-255 and save it as the fourth band of the original image to obtain four-channel data in RGB+D format.

[0016] In box 104, method 100 can output recognition results based on four-channel data from a trained object detection model. The first convolutional layer of the object detection model has four input channels, and all channels are read when the data is loaded. The recognition results are marked with the target box corresponding to the grasping structure and the grasping target within the target box.

[0017] In this embodiment, the target detection model can be YOLO11. This application modifies the network of this model by changing the input channels of the first convolutional kernel in the YOLO11 backbone to 4, enabling it to support four-band data processing. This allows the trained target detection model to process four-channel data and output recognition results. The recognition results will, based on the four-channel data, mark the target bounding box in the region where the grasping structure is located, and label each grasping target within the bounding box. Generally, to ensure recognition effectiveness, both grasping claws on both sides of the grasping structure need to be visible in the image, rather than only one side, to avoid some grasping targets being occluded by other grasping targets in the image. Since cameras typically capture multiple images continuously for processing, the target detection model can process only images where both grasping claws are visible.

[0018] In box 106, method 100 can determine the average depth of the grasping target in this grasping operation based on the depth value of each grasping target, and remove grasping targets whose deviation from the average depth is greater than a ratio threshold, and count the remaining grasping targets.

[0019] In this embodiment, due to the complexity of the target scene, there may be other interfering targets around the grasping structure, causing the identified target box to include a small portion of these interfering targets within its vicinity. Therefore, after determining the target box and filtering out targets outside the target box, further filtering of the grasping targets within the target box is required. Specifically, the depth values ​​of the truly grasped targets should be similar, while the depth values ​​of interfering targets in the surrounding environment will show significant differences because they are not actually in the same location. Furthermore, most of the grasping targets within the target box should be the truly grasped targets. Therefore, the average depth of this grasping operation can be determined directly based on the depth values ​​of each grasping target. As an example, the average depth can be obtained by directly calculating the average of all depth values, or by determining the median of each depth value and using that median as the average depth, or by dividing each depth value according to a preset depth region range and using the median of the depth region range with the most divisions as the average depth, and so on. After determining the average depth, the deviation ratio between the depth value of each grasped target and the average depth is calculated. The deviation ratio can be calculated by dividing the difference between the depth value and the average depth by the average depth. A pre-set ratio threshold (e.g., 20%) is set, and grasped targets with a deviation ratio greater than the threshold are removed. The remaining grasped targets are then counted. Each identified grasped target is labeled in the identification results, so the count is completed based on the number of remaining labeled targets.

[0020] In one possible implementation, determining the average depth of the grasping target in this grasping operation based on the depth values ​​of each grasping target includes: Based on the depth value of the target being grasped, a depth level is determined for each target, and each depth level is divided based on a preset gradient value; and Determine the target depth level with the most targets to be captured, and then determine the average depth of the targets in this capture operation based on the depth values ​​within each target depth level.

[0021] In this embodiment, the depth values ​​can be divided into depth levels (e.g., 1-50, 51-100, etc.) based on a preset gradient value (e.g., 50). Next, the depth level to which the depth value of each grasping target belongs is determined. After dividing all depth values, the level with the highest number of grasping targets is selected as the target depth level, and the average depth is determined based on the depth values ​​within that target depth level. As an example, the average depth can be the average value, median value, etc., of the depth values ​​within the level.

[0022] In one possible implementation, determining the average depth of the target to be grasped during the current grasping operation based on the depth values ​​within the target depth range includes: Calculate the average value of each depth value within the target depth range, and use the average value as the average depth of the target during this grasping operation.

[0023] In this embodiment, the average value of each depth value within the target depth level will be calculated and used as the average depth of each target actually captured in this capture.

[0024] In one possible implementation, the method further includes: Based on historical image data, training samples are determined, which include four-channel data samples and recognition result samples. Based on four-channel data samples, the initial target detection model generates predicted recognition results; and Using the identified sample as a supervision signal, the initial target detection model is trained for at least one round to obtain the target detection model.

[0025] In this embodiment, Figure 2 A schematic diagram of the principle process 200 of the target detection model in the training phase of some embodiments of this disclosure is shown. Historical image data can store images of several targets captured by a grasping device in the past. A depth map of the historical image data is determined using a monocular depth estimation model to construct a four-channel data sample 210-1. After labeling the four-channel data sample 210-1 to obtain the recognition result sample 210-2, training sample 210 can be obtained. The ratio of the training set to the validation set in training sample 210 can be 8:2. The recognition result sample 210-2 can be labeled using the X-AnyLabeling open-source tool, and the labeled categories need to include the target's cross-section and grasping structure. The initial target detection model used for training can be a YOLO11 model, etc. During the training of the target detection model 220 using training sample 210, the model generator 221 can generate a predicted recognition result 222 based on the four-channel data sample 210-1. Based on the comparison between the recognition result sample 210-2 and the predicted recognition result 222, the generator loss 223 can be obtained. The generator loss 223 is then used to determine the loss of the predicted recognition result 222, which can be determined using a comparison loss function. The generator loss 223 can be a relatively large value, and it can then be backpropagated to generator 221 to guide the optimization of generator 221's parameters, achieving one round of supervised training for generator 221. This training process can be iterated round after round until generator 221 can generate a more accurate predicted recognition result 222, i.e., until the loss value calculated by the loss function is smaller. After training, the object detection model 220 can output the recognition result 230.

[0026] In one possible implementation, determining training samples based on historical image data includes: Based on historical image data, original image samples are obtained, and the corresponding depth map samples of the original image samples are determined based on a monocular depth estimation model. Both sides of the grasping claws of the grasping structure are visible in the original image samples. The normalized depth map sample is saved as the fourth band of the original image sample to obtain a four-channel data sample. The target bounding boxes of the grasping structure in the four-channel data samples are labeled, and the cross-sections of the grasped targets within the target bounding boxes are labeled to obtain the recognition result samples; and Training samples are constructed based on four-channel data samples and recognition result samples.

[0027] In this embodiment, since targets such as wood and steel vary in thickness and length, and the grasping process cannot guarantee that the cross-sections of all targets are on the same plane, if the camera angle is not directly facing the grasping structure and cannot clearly see both sides of the grasping structure, the targets can easily be obscured. Therefore, the selected original image samples need to be visible from both sides of the grasping claws of the grasping structure. In addition, to reduce the influence of environmental factors, the labeled data should include as many different weather conditions as possible, such as daytime, nighttime, foggy days, rainy days, and backlighting. At the same time, it is also necessary to ensure the balance of the number of samples for different grasping structures such as gantry crane grabs and forklift grabs, so that the trained samples are more robust. Finally, these original image samples and their corresponding depth map samples will be used to construct four-channel data samples, and the four-channel data samples and the labeled recognition result samples will be used to construct training samples.

[0028] In one possible implementation, after determining the training samples based on historical image data, the method further includes: The training samples are preprocessed, including Mosaic data augmentation, rotation processing, and color perturbation.

[0029] In this embodiment, the training samples can also be preprocessed to further enrich the sample data. For example, the Mosaic data augmentation method can be used. Four images are stitched together into one image by randomly scaling, cropping, and arranging them. This can greatly enrich the detection dataset, add many small targets, and reduce GPU memory usage. Images can also be randomly rotated, such as horizontally flipped, vertically flipped, or rotated 90 degrees. Color perturbations can also be applied to the images to change their brightness and other aspects.

[0030] In one possible implementation, the initial object detection model is trained based on stochastic gradient descent, and Mosaic data augmentation is removed in the final preset training rounds.

[0031] In this embodiment, stochastic gradient descent can be used during model training, for example, for 100 epochs, with the results validated on a validation set after each epoch. Mosaic data augmentation can be enabled for the first 90 epochs to enhance small object detection, while it can be disabled for the last 10 epochs to stabilize model training and help the model better adjust weights to adapt to real-world scenarios. Furthermore, Automatic Mixed Precision (AMP) can be used during training to combine single-precision floating-point numbers (FP32) and half-precision floating-point numbers (FP16) during deep learning training, thereby improving training speed, reducing memory usage, and enhancing numerical stability.

[0032] Figure 3 Schematic diagrams of a port timber intelligent counting device 300 based on depth estimation, according to some embodiments of this disclosure, are shown. The various embodiments in this specification are described in a progressive manner; similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments. Figure 3 As shown, the device 300 includes an image stitching module 301, configured to acquire the original image of the target scene in response to a grasping operation completion command, and determine the depth map corresponding to the original image based on a monocular depth estimation model, so as to stitch the original image and the depth map into four-channel data; a model prediction module 302, configured to output the recognition result from a trained target detection model based on the four-channel data, wherein the input channels of the first convolutional layer of the target detection model are four, and all channels are read when the data is loaded, and the recognition result is marked with the target box corresponding to the grasping structure and the grasping target within the target box; and a target counting module 303, configured to determine the average depth of the grasping target in this grasping operation based on the depth value of each grasping target, and remove grasping targets whose deviation from the average depth is greater than a threshold, and count the remaining grasping targets.

[0033] In one possible implementation, the target counting module 303 is further configured to determine the depth level corresponding to each grasping target based on the depth value of the grasping target, and each depth level is divided based on a preset gradient value; and to determine the target depth level with the most grasping targets, so as to determine the average depth of the grasping target in this grasping operation based on the depth values ​​within the target depth level.

[0034] In one possible implementation, the target counting module 303 is further configured to calculate the average value of each depth value within the target depth level, so as to use the average value as the average depth of the grasping target in this grasping operation.

[0035] In one possible implementation, the apparatus further includes a model training module configured to determine training samples based on historical image data, the training samples including four-channel data samples and recognition result samples; generate predicted recognition results from an initial target detection model based on the four-channel data samples; and perform at least one round of model training on the initial target detection model using the recognition result samples as supervision signals to obtain a target detection model.

[0036] In one possible implementation, the model training module is further configured to acquire original image samples based on historical image data, and determine the corresponding depth map samples of the original image samples based on a monocular depth estimation model, wherein both sides of the grasping claws of the grasping structure are visible in the original image samples; the normalized depth map samples are saved as the fourth band of the original image samples to obtain four-channel data samples; the grasping structure in the four-channel data samples is labeled with target boxes, and the cross-section of the grasping target within the target boxes is labeled to obtain recognition result samples; and training samples are constructed based on the four-channel data samples and the recognition result samples.

[0037] In one possible implementation, the model training module is further configured to preprocess the training samples, including Mosaic data augmentation, rotation processing, and color perturbation.

[0038] In one possible implementation, the initial object detection model is trained based on stochastic gradient descent, and Mosaic data augmentation is removed in the final preset training rounds.

[0039] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this specification is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0040] Figure 4 A block diagram of an electronic device 400 that can implement various embodiments of the present disclosure is shown. For example... Figure 4 As shown, the electronic device 400 includes a processor 410, a disk drive 420, an input / output interface 430, a network interface 440, and a memory 450. The processor 410, disk drive 420, input / output interface 430, network interface 440, and memory 450 can communicate with each other via a communication bus 460.

[0041] The processor 410 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0042] The memory 450 can be implemented in the form of ROM (Read Only Memory), RAM (Read Access Memory), static memory, dynamic storage devices, etc. The memory 450 can store the operating system 451 used to control the operation of the electronic device 400, and the basic input / output system (BIOS) 452 used to control the low-level operations of the electronic device 400. Additionally, it can store a web browser 453, a data storage management system 454, etc. In summary, when the technical solution provided in this application is implemented through software or firmware, the relevant program code is stored in the memory 450 and is called and executed by the processor 410.

[0043] Input / output interface 430 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0044] Network interface 440 is used to connect a communication module (not shown in the figure) to enable communication and interaction between the device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0045] Bus 460 includes a pathway for transmitting information between various components of the device, such as processor 410, disk drive 420, input / output interface 430, network interface 440, and memory 450.

[0046] It should be noted that although the above-described device only shows the processor 410, disk drive 420, input / output interface 430, network interface 440, memory 450, bus 460, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the method of this application, and does not necessarily include all the components shown in the figures.

[0047] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0048] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. Furthermore, although operations are depicted in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0049] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for intelligent counting of port timber based on depth estimation, characterized in that, The method includes: In response to the capture operation completion command, the original image of the target scene is acquired, and the depth map corresponding to the original image is determined based on the monocular depth estimation model, so as to stitch the original image and the depth map into four-channel data; Based on the four-channel data, the trained target detection model outputs the recognition result. The first convolutional layer of the target detection model has four input channels, and all channels are read when the data is loaded. The recognition result is marked with the target box corresponding to the grasping structure and the grasping target within the target box. The average depth of each grasping target in this grasping operation is determined based on the depth value of each grasping target. After removing grasping targets whose deviation from the average depth is greater than a threshold, the remaining grasping targets are counted.

2. The intelligent counting method for port timber based on depth estimation according to claim 1, characterized in that, Determining the average depth of the grasped target in this grasping operation based on the depth values ​​of each grasped target includes: Based on the depth value of the grasping target, a depth level corresponding to each grasping target is determined, and each depth level is divided based on a preset gradient value; and The target depth level with the most grasped targets is determined, and the average depth of the grasped targets in this grasping operation is determined based on the depth values ​​within the target depth level.

3. The intelligent counting method for port timber based on depth estimation according to claim 2, characterized in that, Determining the average depth of the target object in this grasping operation based on the depth values ​​within the target depth range includes: Calculate the average value of each depth value within the target depth range, and use the average value as the average depth of the grasping target in this grasping operation.

4. The intelligent counting method for port timber based on depth estimation according to claim 1, characterized in that, The method further includes: Based on historical image data, training samples are determined, which include four-channel data samples and recognition result samples. Based on the four-channel data samples, the initial target detection model generates predicted recognition results; and Using the identified result samples as supervision signals, the initial target detection model is trained for at least one round to obtain the target detection model.

5. The intelligent counting method for port timber based on depth estimation according to claim 4, characterized in that, The process of determining training samples based on historical image data includes: Based on historical image data, original image samples are obtained, and the corresponding depth map samples of the original image samples are determined based on a monocular depth estimation model. Both sides of the grasping claws of the grasping structure are visible in the original image samples. The normalized depth map sample is saved as the fourth band of the original image sample to obtain a four-channel data sample. The target bounding boxes of the grasping structures in the four-channel data samples are labeled, and the cross-sections of the grasping targets within the target bounding boxes are labeled to obtain recognition result samples; and Training samples are constructed based on the four-channel data samples and the recognition result samples.

6. The intelligent counting method for port timber based on depth estimation according to claim 4, characterized in that, After determining the training samples based on historical image data, the process also includes: The training samples are preprocessed, including Mosaic data augmentation, rotation processing, and color perturbation.

7. The intelligent counting method for port timber based on depth estimation according to claim 4, characterized in that, The initial target detection model is trained based on stochastic gradient descent, and Mosaic data augmentation is removed in the final preset training rounds.

8. A port timber intelligent counting device based on depth estimation, characterized in that, The device includes: The image stitching module is configured to respond to the capture operation completion command, acquire the original image of the target scene, determine the depth map corresponding to the original image based on the monocular depth estimation model, and stitch the original image and the depth map into four-channel data. The model prediction module is configured to output recognition results from the trained object detection model based on the four-channel data. The first convolutional layer of the object detection model has four input channels, and all channels are read when the data is loaded. The recognition results are marked with the target box corresponding to the grasping structure and the grasping target within the target box. The target counting module is configured to determine the average depth of the target in the current grasping operation based on the depth value of each target, and to remove targets whose deviation from the average depth is greater than a threshold value, and then count the remaining targets.

9. An electronic device, characterized in that, include: One or more processors, and A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the port timber intelligent counting method based on depth estimation as described in any one of claims 1-7.

10. A computer program product, characterized in that, The system includes a computer program that, when executed by a processor, implements a port timber intelligent counting method based on depth estimation according to any one of claims 1-7.

Images