Lock counting method, apparatus, device, and storage medium

Abstract

The present application relates to the technical field of artificial intelligence, and more particularly to a lock counting method, device, equipment and storage medium, wherein whether an automated guided vehicle for carrying a lock to be counted reaches a lock counting station is detected; if the automated guided vehicle reaches the lock counting station, an image acquisition module is used to acquire an image of the lock to be counted carried by the automated guided vehicle; a lock detection model is used to detect the lock image to obtain a lock bounding box indicating a lock individual and a lock category; and counting and statistics are performed according to the lock bounding box and the lock category to obtain a lock counting result. Compared with the prior art, the present application can improve the efficiency and accuracy of lock counting.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a lock counting method, a lock counting device, a computer device, and a computer-readable storage medium. Background Technology

[0002] Locks, as sealing devices with sealing, protection, and access control as their core functions, have existed since the early days of human civilization, accompanying the need for private property. Over thousands of years of evolution, they have developed from simple mechanical devices into comprehensive security systems integrating physical protection, electronic authentication, and intelligent interaction, widely used in various scenarios such as daily life, commerce, public safety, and industry. In the lock manufacturing process, both the entry and exit of locks require counting. Only by accurately counting locks can the accuracy of lock entry and exit be ensured. In related technologies, manual counting is commonly used. However, this manual counting method is not only inefficient but also prone to counting errors due to factors such as visual fatigue. Summary of the Invention

[0003] This invention provides a lock counting method, a lock counting device, a computer device, and a computer-readable storage medium, which can improve the efficiency and accuracy of lock counting.

[0004] On one hand, the lock counting method provided by the present invention includes: Detect whether the automated guided vehicle used to transport the locks to be counted has arrived at the lock counting station; If the automated guided vehicle arrives at the lock counting station, the image acquisition module will capture images of the locks to be counted carried by the automated guided vehicle to obtain lock images. Lock detection is performed on lock images using a lock detection model to obtain the lock bounding box and lock category indicating individual locks; The lock count is obtained by counting and statistically analyzing the lock bounding box and lock category.

[0005] Optionally, in one embodiment, an image acquisition module acquires images of the locks to be counted carried by the automated guided vehicle to obtain lock images, including: Get the current pallet size of the lock-carrying pallet of the automated guided vehicle; Based on the correspondence between the pallet size and the shooting distance associated with the automated guided vehicle, determine the target shooting distance corresponding to the current pallet size; The target shooting distance is the shooting distance between the driving image acquisition module and the lock carrier tray. The image acquisition module acquires images of the locks to be counted in the lock carrier tray to obtain lock images.

[0006] Optionally, in one embodiment, an image acquisition module acquires images of the locks to be counted in the lock transport tray to obtain lock images, including: The image acquisition module acquires images of the locks to be counted in the lock carrier tray, resulting in images of ordinary locks, infrared locks, and depth locks.

[0007] Optionally, in one embodiment, lock detection is performed on the lock image using a lock detection model to obtain a lock bounding box indicating the individual lock and the lock category, including: The images of ordinary locks, infrared locks, and depth locks are merged by channel to obtain a channel-merged image; The first lock detection model is used to detect locks in the merged channel image, resulting in lock bounding boxes and lock categories that indicate individual locks.

[0008] Optionally, in one embodiment, lock detection is performed on the channel-merged image using a first lock detection model to obtain lock bounding boxes indicating individual locks and lock categories, including: Lock detection is performed on the channel-merged image using the first lock detection model to obtain candidate lock bounding boxes indicating individual locks; The second lock detection model is used to extract features from ordinary lock images, infrared lock images and depth lock images respectively, to obtain texture feature maps of ordinary lock images, temperature distribution feature maps of infrared lock images and spatial structure feature maps of depth lock images; The overlapping region of the locks is determined based on the candidate lock bounding boxes, and the texture feature map, temperature distribution feature map and spatial structure feature map are fused based on the overlapping region of the locks to obtain the fused feature map; Based on the fused feature map, lock detection is performed using a second lock detection model to obtain the lock bounding box and lock category indicating the individual lock.

[0009] Optionally, in one embodiment, a fused feature map is obtained by fusing the texture feature map, temperature distribution feature map, and spatial structure feature map of the overlapping area of ​​the lock, including: For the overlapping areas of the locks, the fusion weights are assigned to the texture feature map, temperature distribution feature map, and spatial structure feature map according to the weight order of spatial structure feature map > temperature distribution feature map > texture feature map. For non-locking overlapping areas, fusion weights are assigned to the texture feature map, temperature distribution feature map, and spatial structure feature map according to the weight order of texture feature map > spatial structure feature map > temperature distribution feature map. Based on the fusion weights of the texture feature map, temperature distribution feature map, and spatial structure feature map in the overlapping area of ​​the lock, and the fusion weights of the texture feature map, temperature distribution feature map, and spatial structure feature map in the non-overlapping area of ​​the lock, the texture feature map, temperature distribution feature map, and spatial structure feature map are weighted and fused to obtain the fused feature map.

[0010] Optionally, in one embodiment, after counting and statistically analyzing the locks based on the lock bounding box and lock category to obtain the lock count result, the method further includes: Obtain the report generation strategy, and generate a lock count report based on the lock count results according to the report generation strategy.

[0011] Secondly, the lock counting device provided by the present invention includes: The station detection module is used to detect whether the automated guided vehicle carrying the locks to be counted has arrived at the lock counting station; The image acquisition module is used to acquire images of the locks to be counted carried by the automated guided vehicle when the vehicle arrives at the lock counting station, and obtain lock images. The lock detection module is used to detect locks in lock images using a lock detection model, and to obtain the lock bounding box and lock category indicating the individual locks. The counting and statistics module is used to perform counting and statistics based on the lock boundary and lock category to obtain the lock count results.

[0012] Thirdly, the computer device provided by the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the lock counting method provided by the present invention.

[0013] Fourthly, the computer-readable storage medium provided by the present invention stores a computer program that, when executed by a processor, implements the lock counting method provided by the present invention.

[0014] This invention provides a lock counting scheme. It detects whether an automated guided vehicle (AGV) carrying the locks to be counted has arrived at the lock counting station. If the AAV arrives, an image acquisition module captures images of the locks being counted, obtaining lock images. A lock detection model then performs lock detection on these images, obtaining lock bounding boxes and lock categories indicating individual locks. Counting is then performed based on the lock bounding boxes and lock categories to obtain the lock count result. Thus, this invention, combined with an AAV, achieves automated lock counting, improving efficiency compared to traditional manual counting. Furthermore, since no manual intervention is required throughout the counting process, it avoids counting errors caused by visual fatigue and improves the accuracy of lock counting. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the lock counting method provided in an embodiment of the present invention; Figure 2 This is a deployment example diagram of the computer equipment and image acquisition module in an embodiment of the present invention; Figure 3 This is a schematic diagram of a lock image acquired in an embodiment of the present invention; Figure 4 This is an example diagram of merging ordinary lock images, infrared lock images, and depth lock images by channel in an embodiment of the present invention; Figure 5 This is an example image of a lock counting image in an embodiment of the present invention; Figure 6 This is a schematic diagram of the lock counting device provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the technical problems solved, the technical solutions, and the beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0018] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0019] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0020] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0021] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0022] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0023] This invention provides a lock counting method, a lock counting device, a computer device, and a storage medium. The lock counting method can be executed by the lock counting device or by a computer device integrating the lock counting device. The method involves detecting whether an automated guided vehicle (AGV) carrying the locks to be counted has arrived at the lock counting station. If the AAV arrives at the lock counting station, an image acquisition module acquires images of the locks being counted carried by the AAV to obtain lock images. A lock detection model is used to detect locks in the lock images to obtain lock bounding boxes and lock categories indicating individual locks. Counting is performed based on the lock bounding boxes and lock categories to obtain the lock counting result.

[0024] Please refer to Figure 1 This is a flowchart illustrating a lock counting method disclosed in an embodiment of the present invention, as shown below. Figure 1 As shown, the process of this lock counting method can be as follows: In step 110, it is detected whether the automated guided vehicle used to transport the locks to be counted has arrived at the lock counting station.

[0025] Automated Guided Vehicles (AGVs) are intelligent devices that transport materials without human drivers, using autonomous navigation technology.

[0026] The image acquisition module is configured to acquire images. There are no specific limitations on the type of images acquired by the image acquisition module, including but not limited to at least one of ordinary images, depth images, and infrared images. In the following embodiments, the image acquisition module is separated from the computer device as an example.

[0027] The lock counting station refers to the position where the computer equipment completes the counting of locks, that is, the position where the computer equipment can acquire images through the image acquisition module.

[0028] The automated guided vehicle is configured to transport the locks to be counted to the lock counting station, so that the computer equipment can acquire images of the locks to be counted carried by the automated guided vehicle through the image acquisition module, and then use the acquired images to count the locks to be counted.

[0029] For example, please refer to Figure 2 The computer equipment and image acquisition module are mounted on a bracket within a semi-enclosed space formed by a shield, which has reserved entrances and exits for automated guided vehicles to enter and exit. Figure 2 As shown, the lock counting station is located directly below the image acquisition module. When the automated guided vehicle (AGV) carries the locks to be counted to the position directly below the image acquisition module via the lock transport tray, the computer equipment will be able to acquire images of the locks to be counted carried by the AGV through the image acquisition module.

[0030] In this embodiment of the invention, the detection by the computer device of whether the automated guided vehicle (AGV) has reached the lock counting station can be either proactive or reactive. For example, taking reactive detection, on the one hand, when the AGV arrives at the lock counting station carrying the locks to be counted according to the configured guidance path, it sends a positioning prompt to the computer device indicating that it has reached the lock counting station; on the other hand, after receiving the positioning prompt from the AGV, the computer device determines that the AGV has reached the lock counting station.

[0031] In step 120, if the automated guided vehicle arrives at the lock counting station, the image acquisition module will capture images of the locks to be counted carried by the automated guided vehicle to obtain lock images.

[0032] As described above, the lock counting station refers to the position where the image acquisition module can acquire images of the locks to be counted carried by the automated guided vehicle (AGV). Accordingly, when the AGV is detected to have arrived at the lock counting station, the computer sends an image acquisition command to the image acquisition module, instructing it to acquire images of the locks to be counted carried by the AGV, which are then recorded as lock images. After acquiring the lock images of the locks to be counted carried by the AGV, the image acquisition module returns these images to the computer. Thus, the computer obtains the lock images of the locks to be counted carried by the AGV through the image acquisition module.

[0033] In 130, the lock image is detected by the lock detection model to obtain the lock bounding box and lock category indicating the individual lock.

[0034] It should be noted that, in this embodiment of the invention, a lock detection model is also pre-trained. This lock detection model is configured to take an image that may include a lock as input and a predicted bounding box and category indicating the lock in the image as output.

[0035] Specifically, labeled sample lock images are used to perform transfer training on the pre-trained target detection model to obtain a lock detection model suitable for lock detection.

[0036] For example, it can be implemented as follows: The YOLOv8-based object detection model is used as the base model for training the lock detection model. Its model structure includes the following components: Backbone network: A backbone network is a network used to extract image features. Its main function is to transform the original input image into a multi-layer feature map for use in subsequent detection tasks.

[0037] Neck network: The neck network is used to combine feature maps from different levels to generate feature maps with multi-scale information, thereby improving detection accuracy.

[0038] The detection head consists of three different output layers, each responsible for detecting targets of different scales: large, medium, and small.

[0039] During the training phase, sample lock images covering different categories and placement methods are collected. Each sample lock image is labeled to obtain a training label including the ground truth bounding box of each lock in the image and the lock category. Each sample lock image and its corresponding training label are used as a set of samples to input into the object detection model for training until a preset stopping condition is met, resulting in a lock detection model suitable for lock detection. The configuration of the preset stopping condition is not specifically limited here. For example, the preset stopping condition can be configured as: loss convergence, or the number of parameter update epochs of the object detection model reaching a preset number of epochs, such as 300 epochs, etc.

[0040] In addition, the sample lock images can be enhanced using configured data augmentation strategies before being used in the training process. There are no specific restrictions on the configuration of data augmentation strategies here, including but not limited to rotation, scaling, and cropping, etc.

[0041] Accordingly, in the embodiments of this application, after the computer device acquires the lock image of the lock to be counted carried by the automated guided vehicle through the image acquisition module, it inputs the lock image into the lock detection model to perform lock detection, and obtains the lock bounding box and lock category output by the lock detection model to indicate the individual locks in the lock image.

[0042] In 140, the lock count is obtained by counting based on the lock bounding box and lock category.

[0043] In this embodiment of the invention, after the computer device detects locks in a lock image using a lock detection model and obtains the lock bounding box and lock category indicating individual locks in the lock image, it further performs counting statistics based on the detected lock bounding boxes and lock categories according to a configured statistical strategy, and obtains the corresponding lock counting result.

[0044] For example, the statistical strategy can be configured to count according to at least one of the following counting methods: The computer equipment counts all the bounding boxes of the locks detected by the lock detection model to obtain the total number of locks to be counted. The computer equipment counts the bounding boxes of different lock categories to obtain the number of locks of each category in the locks to be counted.

[0045] For example, for the collected images of locks to be counted, the computer device counts a total of 15 locks to be counted, of which 10 are of type A and 5 are of type B.

[0046] Optionally, in one embodiment, an image acquisition module acquires images of the locks to be counted carried by the automated guided vehicle to obtain lock images, including: Get the current pallet size of the lock-carrying pallet of the automated guided vehicle; Based on the correspondence between the pallet size and the shooting distance associated with the automated guided vehicle, determine the target shooting distance corresponding to the current pallet size; The target shooting distance is the shooting distance between the driving image acquisition module and the lock carrier tray. The image acquisition module acquires images of the locks to be counted in the lock carrier tray to obtain lock images.

[0047] In this embodiment of the invention, the automated guided vehicle (AGV) transports the locks to be counted via a lock transport tray. The shape and color of the lock transport tray are not specifically limited here. For example, this embodiment uses a rectangular shape, made of plastic, and with a color significantly different from the color of the locks to be counted. Furthermore, to accommodate lock counting tasks with varying numbers of locks, this embodiment provides lock transport trays of different sizes.

[0048] In addition, the image acquisition module is mounted on a lifting mechanism that can drive the image acquisition module to move vertically up and down, thereby changing the shooting distance between the image acquisition module and the lock carrier tray.

[0049] It is understandable that for the same lock carrier tray, the closer the shooting distance between the image acquisition module and the lock carrier tray, the larger the proportion of the lock carrier tray in the image acquired by the image acquisition module. To enable the image acquisition module to acquire images more suitable for lock counting by the computer equipment, for automated guided vehicles (AGVs) with different carrying heights, this embodiment of the invention pre-establishes a correspondence between the tray size and the shooting distance associated with each AAV. Specifically, for an AAV with a certain carrying height, the correspondence between the tray size and the shooting distance associated with that AAV describes the shooting distance at which the image of each size of lock carrier tray exactly fills the imaging range of the image acquisition module, for lock carrier trays of different sizes.

[0050] Correspondingly, when capturing images of the locks to be counted carried by the automated guided vehicle (AGV) using the image acquisition thumb, the computer first sends a size query request to the AGV, requesting it to return the current pallet size of the lock transport tray used to transport the locks to be counted. After receiving the current pallet size returned by the AGV, the computer further determines the shooting distance corresponding to the current pallet size based on the correspondence between the pallet size and the shooting distance associated with the AGV, and records this as the target shooting distance. Finally, a drive command is sent to the lifting mechanism, instructing it to drive the image acquisition module to perform a vertical lifting movement until the shooting distance between the image acquisition module and the lock transport tray is the target shooting distance. Then, the image acquisition module performs image acquisition, thereby acquiring the lock image of the locks to be counted. The image edge content of this lock image is the pallet edge of the lock transport tray. Figure 3 As shown.

[0051] Optionally, in one embodiment, an image acquisition module acquires images of the locks to be counted in the lock transport tray to obtain lock images, including: The image acquisition module acquires images of the locks to be counted in the lock carrier tray, resulting in images of ordinary locks, infrared locks, and depth locks.

[0052] In this embodiment of the invention, the image acquisition module consists of a regular camera, an infrared camera, and a depth camera, enabling the image acquisition module to simultaneously acquire regular images, infrared images, and depth images.

[0053] Ordinary images are common color images that present color information using the pixel values ​​of individual pixels, such as RGB images. Each pixel consists of three color channels: red, green, and blue, which can display rich colors and texture details and intuitively reflect the appearance characteristics of objects, such as shape, color, and surface patterns.

[0054] Infrared images are usually presented as grayscale or pseudo-color images (e.g., red represents high temperature and blue represents low temperature) to visually reflect the temperature distribution of an object, but they cannot reflect the texture and color of the object.

[0055] Depth images are typically presented in grayscale form. Depth information is usually encoded as grayscale values ​​in the depth image. The grayscale level of each pixel represents the distance to that point. This representation makes the depth image visually similar to a black and white photograph, but each pixel actually stores distance information rather than color information.

[0056] Accordingly, in this embodiment of the invention, the computer device will acquire ordinary images, infrared images and depth images of the locks to be counted in the lock carrier tray through the image acquisition module, and record them as ordinary lock images, infrared lock images and depth lock images, respectively.

[0057] Optionally, in one embodiment, lock detection is performed on the lock image using a lock detection model to obtain a lock bounding box indicating the individual lock and the lock category, including: The images of ordinary locks, infrared locks, and depth locks are merged by channel to obtain a channel-merged image; The first lock detection model is used to detect locks in the merged channel image, resulting in lock bounding boxes and lock categories that indicate individual locks.

[0058] In this embodiment of the invention, the lock detection model includes a first lock detection model, which is obtained by training a sample channel merged image as input and training labels corresponding to the sample channel merged image as output. The sample channel merged image is obtained by merging matched sample ordinary lock images, sample infrared lock images, and sample depth lock images by channel after alignment. The training labels are used to indicate the true bounding box and lock category of individual locks in the sample ordinary lock image / sample infrared lock image / sample depth lock image.

[0059] When detecting locks using a lock detection model, the computer first aligns the ordinary lock image, the infrared lock image, and the depth lock image. Then, it merges these aligned images by channel to obtain a channel-merged image. For example, please refer to... Figure 4 A standard lock image is an RGB three-channel image, while an infrared lock image and a depth lock image are both single-channel images. Therefore, the channel-merged image obtained by merging channels will include five channels.

[0060] After merging the channels to obtain the merged image, the computer device further inputs the merged channel image into the first lock detection model to detect locks, and obtains the lock bounding box and lock category indicating the individual lock.

[0061] Optionally, in one embodiment, lock detection is performed on the channel-merged image using a first lock detection model to obtain lock bounding boxes indicating individual locks and lock categories, including: Lock detection is performed on the channel-merged image using the first lock detection model to obtain candidate lock bounding boxes indicating individual locks; The second lock detection model is used to perform feature requirements on ordinary lock images, infrared lock images and depth lock images respectively, to obtain texture feature maps of ordinary lock images, temperature distribution feature maps of infrared lock images and spatial structure feature maps of depth lock images; The overlapping region of the locks is determined based on the candidate lock bounding boxes, and the texture feature map, temperature distribution feature map and spatial structure feature map are fused based on the overlapping region of the locks to obtain the fused feature map; Based on the fused feature map, lock detection is performed using a second lock detection model to obtain the lock bounding box and lock category indicating the individual lock.

[0062] In this embodiment of the invention, the lock detection model also includes a second lock detection model. Unlike the above embodiments, this embodiment of the invention does not directly use the lock detection result of the first lock detection model as the final detection result, but uses it as a reference for the second lock detection model to perform lock detection.

[0063] It should be noted that the second lock detection model includes three feature extraction branches, which are respectively suitable for extracting texture features, temperature distribution features, and spatial structure features. Each feature extraction branch has the same structure but does not share parameters. Each feature extraction branch consists of a backbone network and a neck network.

[0064] When performing lock detection, the computer equipment first performs lock detection on the channel merged image using the first lock detection model to obtain candidate bounding boxes indicating individual locks.

[0065] Then, the computer device extracts the texture feature map of the ordinary lock image through the feature extraction branch in the second lock detection model that is suitable for extracting texture features, extracts the temperature distribution feature map of the infrared lock image through the feature extraction branch in the second lock detection model that is suitable for extracting temperature distribution features, and extracts the spatial structure feature map of the depth lock image through the feature extraction branch in the second lock detection model that is suitable for extracting spatial structure features.

[0066] Then, the computer device determines the overlapping region of the locks based on the candidate lock bounding boxes detected by the first lock detection model. For example, the computer device can identify overlapping candidate lock bounding boxes, then determine the minimum bounding rectangle region of the overlapping candidate lock bounding boxes, and determine the overlapping region of the locks based on the minimum bounding rectangle region. For example, the minimum bounding rectangle region can be directly determined as the overlapping region of the locks, or the minimum bounding rectangle region can be enlarged according to a preset scaling ratio and used as the overlapping region of the locks.

[0067] Then, based on the determined overlapping areas of the locks, the computer device fuses texture feature maps, temperature distribution feature maps, and spatial structure feature maps according to the configured feature fusion strategy to obtain a fused feature map. Finally, this fused feature image is input into the detection head of the second lock detection model to obtain the lock bounding box and lock category indicating the individual lock. No specific restrictions are placed on the configuration of the feature fusion strategy here.

[0068] Optionally, in one embodiment, a fused feature map is obtained by fusing the texture feature map, temperature distribution feature map, and spatial structure feature map of the overlapping area of ​​the lock, including: For the overlapping areas of the locks, the fusion weights are assigned to the texture feature map, temperature distribution feature map, and spatial structure feature map according to the weight order of spatial structure feature map > temperature distribution feature map > texture feature map. For non-locking overlapping areas, fusion weights are assigned to the texture feature map, temperature distribution feature map, and spatial structure feature map according to the weight order of texture feature map > spatial structure feature map > temperature distribution feature map. Based on the fusion weights of the texture feature map, temperature distribution feature map, and spatial structure feature map in the overlapping area of ​​the lock, and the fusion weights of the texture feature map, temperature distribution feature map, and spatial structure feature map in the non-overlapping area of ​​the lock, the texture feature map, temperature distribution feature map, and spatial structure feature map are weighted and fused to obtain the fused feature map.

[0069] This invention provides an optional feature fusion strategy, in which a weighted fusion method is used to fuse texture feature maps, temperature distribution feature maps, and spatial structure feature maps, and the weights of texture feature maps, temperature distribution feature maps, and spatial structure feature maps are different in the overlapping areas of locks and in the non-overlapping areas of locks.

[0070] In the overlapping areas of the locks, the texture features of the locks will be lost due to the overlap. However, the spatial structure of the locks is continuous, so even if there is overlap, the spatial structure of adjacent areas can still be used for inference. In addition, considering the temperature difference between the locks and the lock carrier tray (the carrier tray is made of plastic, and the locks are made of metal), the temperature distribution features are used to further supplement the evidence of the existence of the locks. Accordingly, when assigning weights to the overlapping areas of the locks, the computer device assigns fusion weights to the texture feature map, temperature distribution feature map, and spatial structure feature map according to the weight relationship of spatial structure feature map > temperature distribution feature map > texture feature map. Here, there are no specific restrictions on the size of the fusion weights assigned to the texture feature map, temperature distribution feature map, and spatial structure feature map respectively.

[0071] Furthermore, for non-locking overlapping areas, since there is no overlap between locks, texture features will directly reflect the presence of locks, while spatial structure features and temperature distribution features will indirectly supplement the evidence of lock presence. Accordingly, when assigning weights to non-locking overlapping areas, the computer device assigns fusion weights to the texture feature map, temperature distribution feature map, and spatial structure feature map according to the weight relationship of texture feature map > spatial structure feature map > temperature distribution feature map. Here, there are no specific restrictions on the size of the fusion weights assigned to the texture feature map, temperature distribution feature map, and spatial structure feature map respectively.

[0072] As shown above, after assigning weights to the spatial structure feature map, temperature distribution feature map, and texture feature map in the overlapping and non-overlapping areas of the lock, the texture feature map, temperature distribution feature map, and spatial structure feature map are further weighted and fused in the overlapping areas of the lock, based on their respective fusion weights in the non-overlapping areas, to obtain the fused feature map.

[0073] Optionally, in one embodiment, after counting and statistically analyzing the locks based on the lock bounding box and lock category to obtain the lock count result, the method further includes: Obtain the report generation strategy, and generate a lock count report based on the lock count results according to the report generation strategy.

[0074] The report generation strategy is used to indicate how to generate a lock counting report using the lock counting results, including the report format, content structure, data filtering criteria, and output format. The specific report generation strategy can be configured by those skilled in the art according to actual needs, and no specific restrictions are imposed here.

[0075] In this embodiment of the invention, after obtaining the lock count results, the computer device further acquires the configured report generation strategy and generates a lock count report based on the lock count results according to the report generation strategy. Furthermore, the computer device can send the generated lock count report to the inventory management system, so that the inventory management system can use the lock count report for inbound and outbound statistics, production planning scheduling, and real-time inventory updates.

[0076] For example, a generated lock count report includes not only the lock count results but also a visualized image of the lock count, such as... Figure 5 As shown, the lock count image is obtained by adding lock bounding boxes to the original acquired lock image.

[0077] As described above, the lock counting scheme provided by this invention detects whether the automated guided vehicle (AGV) carrying the locks to be counted has arrived at the lock counting station. If the AGV arrives at the lock counting station, an image acquisition module acquires images of the locks being counted carried by the AGV, obtaining lock images. A lock detection model then performs lock detection on the lock images, obtaining lock bounding boxes and lock categories indicating individual locks. Counting is then performed based on the lock bounding boxes and lock categories to obtain the lock counting result. Thus, this invention, combined with an AGV, achieves automated lock counting, improving efficiency compared to traditional manual counting. Furthermore, since no manual intervention is required throughout the counting process, counting errors caused by visual fatigue are avoided, and the accuracy of lock counting is also improved.

[0078] To facilitate better implementation of the above lock counting method, this embodiment of the invention also provides a corresponding lock counting device. The meanings of the terms used are the same as in the lock counting method described above; for specific implementation details, please refer to the descriptions in the above method embodiments.

[0079] Please refer to Figure 6 The lock counting device may include a behavior station detection module 210, an image acquisition module 220, a lock detection module 230, and a counting and statistics module 240. Detailed descriptions of each functional module are as follows: The station detection module 210 is used to detect whether the automatic guided transport vehicle carrying the locks to be counted has arrived at the lock counting station; The image acquisition module 220 is used to acquire images of the locks to be counted carried by the automated guided vehicle when the automated guided vehicle arrives at the lock counting station, and obtain lock images. The lock detection module 230 is used to detect locks in lock images using a lock detection model, and to obtain the lock bounding box and lock category indicating individual locks. The counting and statistics module 240 is used to perform counting and statistics based on the lock boundary frame and lock category to obtain the lock counting results.

[0080] Optionally, in one embodiment, the image acquisition module 220 is used to obtain the current pallet size of the lock transport pallet of the automated guided vehicle; determine the target shooting distance corresponding to the current pallet size according to the correspondence between the pallet size associated with the automated guided vehicle and the shooting distance; drive the shooting distance between the image acquisition module and the lock transport pallet as the target shooting distance; and acquire images of the locks to be counted in the lock transport pallet through the image acquisition module to obtain lock images.

[0081] Optionally, in one embodiment, the image acquisition module 220 is used to acquire images of the locks to be counted in the lock carrier tray through the image acquisition module, and obtain ordinary lock images, infrared lock images and depth lock images.

[0082] Optionally, in one embodiment, the lock detection module 230 is used to merge the ordinary lock image, the infrared lock image, and the depth lock image by channel to obtain a channel-merged image; and to perform lock detection on the channel-merged image using a first lock detection model to obtain a lock bounding box indicating the individual lock and the lock category.

[0083] Optionally, in one embodiment, the lock detection module 230 is used to perform lock detection on the channel-merged image using a first lock detection model to obtain candidate lock bounding boxes indicating individual locks; to extract features from the ordinary lock image, infrared lock image, and depth lock image using a second lock detection model to obtain a texture feature map of the ordinary lock image, a temperature distribution feature map of the infrared lock image, and a spatial structure feature map of the depth lock image; to determine the lock overlapping region based on the candidate lock bounding boxes, and to fuse the texture feature map, temperature distribution feature map, and spatial structure feature map based on the lock overlapping region to obtain a fused feature map; and to perform lock detection using the second lock detection model based on the fused feature map to obtain lock bounding boxes indicating individual locks and lock categories.

[0084] Optionally, in one embodiment, the lock detection module 230 is used to assign fusion weights to the texture feature map, temperature distribution feature map, and spatial structure feature map respectively for the lock overlapping area according to the weight relationship of spatial structure feature map > temperature distribution feature map > texture feature map; for the non-lock overlapping area, the fusion weights are assigned to the texture feature map, temperature distribution feature map, and spatial structure feature map respectively according to the weight relationship of texture feature map > spatial structure feature map > temperature distribution feature map; and the texture feature map, temperature distribution feature map, and spatial structure feature map are weighted and fused according to their respective fusion weights in the lock overlapping area and in the non-lock overlapping area to obtain a fused feature map.

[0085] Optionally, in one embodiment, the counting and statistics module 240 is used to obtain a report generation strategy and generate a lock counting report based on the lock counting results according to the report generation strategy.

[0086] Specific limitations regarding the lock counting device can be found in the limitations of the lock counting method described above, and will not be repeated here. Each module in the aforementioned lock counting device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0087] In one embodiment, a computer device is provided, which may be a wireless network access device, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface connects to external wireless clients, providing wireless network access services to the connected clients. When executed by the processor, the computer program implements the lock counting method provided by this invention.

[0088] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the lock counting method described above.

[0089] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the lock counting method described above.

[0090] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0091] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0092] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

[0093] It should be noted that when the above embodiments of the present invention are applied to specific products or technologies, and user-related data is involved, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

Claims

1. A method for counting locks, characterized in that, include: Detect whether the automated guided vehicle used to transport the locks to be counted has arrived at the lock counting station; If the automated guided vehicle (AGV) arrives at the lock counting station, the current pallet size of the lock transport tray of the AGV is obtained; based on the correspondence between the pallet size associated with the AGV and the shooting distance, a target shooting distance corresponding to the current pallet size is determined; wherein, the target shooting distance is the shooting distance at which the image of the lock transport tray just fills the imaging range of the image acquisition module; the shooting distance between the image acquisition module and the lock transport tray is the target shooting distance, and the image acquisition module performs image acquisition on the locks to be counted in the lock transport tray to obtain ordinary lock images, infrared lock images, and depth lock images; The ordinary lock image, the infrared lock image, and the depth lock image are merged by channel to obtain a channel-merged image; Lock detection is performed on the merged channel image using a first lock detection model to obtain candidate lock bounding boxes indicating individual locks. Feature extraction is then performed on the ordinary lock image, the infrared lock image, and the depth lock image using a second lock detection model to obtain texture feature maps for the ordinary lock image, temperature distribution feature maps for the infrared lock image, and spatial structure feature maps for the depth lock image. Overlapping lock regions are determined based on the candidate lock bounding boxes, and for these regions, fusion weights are assigned to the texture feature map, temperature distribution feature map, and spatial structure feature map according to the weight order: spatial structure feature map > temperature distribution feature map > texture feature map. For non-overlapping lock regions… In each region, fusion weights are assigned to the texture feature map, temperature distribution feature map, and spatial structure feature map according to the weight order: texture feature map > spatial structure feature map > temperature distribution feature map. Based on the fusion weights of each feature map in the overlapping lock region and in the non-overlapping lock region, the texture feature map, temperature distribution feature map, and spatial structure feature map are weighted and fused to obtain a fused feature map. Based on the fused feature map, lock detection is performed using the second lock detection model to obtain the lock bounding box and lock category indicating the individual lock. The lock count is obtained by counting and statistically analyzing the lock bounding box and the lock category.

2. The lock counting method according to claim 1, characterized in that, After obtaining the lock count result by counting and statistically analyzing the lock bounding box and the lock category, the method further includes: Obtain the report generation strategy, and generate a lock count report based on the lock count results according to the report generation strategy.

3. A lock counting device, characterized in that, include: The station detection module is used to detect whether the automated guided vehicle carrying the locks to be counted has arrived at the lock counting station; An image acquisition module is used to: acquire the current pallet size of the lock transport tray of the automated guided vehicle when it arrives at the lock counting station; determine a target shooting distance corresponding to the current pallet size based on the correspondence between the pallet size associated with the automated guided vehicle and the shooting distance; wherein the target shooting distance is the shooting distance that makes the image of the lock transport tray just fill the imaging range of the image acquisition module; drive the shooting distance between the image acquisition module and the lock transport tray as the target shooting distance; and acquire images of the locks to be counted in the lock transport tray through the image acquisition module to obtain ordinary lock images, infrared lock images, and depth lock images; merge the ordinary lock images, the infrared lock images, and the depth lock images by channel to obtain a channel-merged image; The lock detection module is used to detect locks in the channel-merged image using a first lock detection model to obtain candidate lock bounding boxes indicating individual locks; to extract features from the ordinary lock image, the infrared lock image, and the depth lock image using a second lock detection model to obtain a texture feature map of the ordinary lock image, a temperature distribution feature map of the infrared lock image, and a spatial structure feature map of the depth lock image; to determine the lock overlap region based on the candidate lock bounding boxes, and for the lock overlap region, to assign fusion weights to the texture feature map, the temperature distribution feature map, and the spatial structure feature map according to the weight relationship of spatial structure feature map > temperature distribution feature map > texture feature map; and to perform feature extraction on non-lock images. In the overlapping areas of the locks, fusion weights are assigned to the texture feature map, temperature distribution feature map, and spatial structure feature map according to the weight order: texture feature map > spatial structure feature map > temperature distribution feature map. Based on the fusion weights of each feature map in the overlapping areas and in the non-overlapping areas, the texture feature map, temperature distribution feature map, and spatial structure feature map are weighted and fused to obtain a fused feature map. Based on the fused feature map, lock detection is performed using the second lock detection model to obtain the lock bounding box and lock category indicating the individual lock. The counting and statistics module is used to perform counting and statistics based on the lock boundary frame and the lock category to obtain the lock counting result.

4. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the lock counting method according to any one of claims 1 to 2.

5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the lock counting method according to any one of claims 1 to 2.

Images