Safety detection method, device and equipment for tray conveying, medium and program product

Abstract

The invention relates to a safety detection method, device and equipment for tray conveying, a medium and a program product. The method comprises the steps that in response to a tray conveying instruction, conveying equipment is controlled to collect a tray image of a to-be-conveyed tray; the tray conveying instruction comprises a conveying task, and the conveying task comprises an empty tray conveying task or a loading tray conveying task; identifying a tray state corresponding to the tray image, wherein the tray state comprises an empty tray or a loading tray; and under the condition that the tray state is not matched with the conveying task, tray abnormal information is output. By the adoption of the method, automatic verification in the tray conveying process is achieved, material conveying abnormity caused by tray type errors is avoided, stacking machine damage caused by the fact that the carrying tray is conveyed to the empty tray stacking machine by mistake is also avoided, and potential safety hazards in the conveying process are reduced.

Description

Technical Field

[0001] The present application relates to the field of safety detection technology, and in particular to a safety detection method, device, equipment, medium and program product for pallet transportation. Background Art

[0002] With the rapid development of intelligent manufacturing and automated logistics technologies, material transportation is gradually being achieved through AGV (Automated Guided Vehicle) in the tobacco production process, improving material turnover efficiency and production coordination.

[0003] However, because AGVs in traditional transportation methods can only execute transport operations along pre-set routes and lack the ability to verify the actual status of materials in real time, mishandling is prone to occur. For example, a loaded pallet could be transported directly to an empty pallet palletizer, damaging the empty palletizer. This not only impacts normal production progress but also increases safety risks. Summary of the Invention

[0004] Based on this, it is necessary to provide a safety detection method, device, equipment, medium and program product for pallet transportation to address the above technical problems.

[0005] In a first aspect, the present application provides a pallet transport safety detection method, comprising:

[0006] In response to a pallet transport instruction, controlling the transport device to capture a pallet image of the pallet to be transported; the pallet transport instruction includes a transport task, and the transport task includes an empty pallet transport task or a loaded pallet transport task;

[0007] Identify the pallet status corresponding to the pallet image, where the pallet status includes an empty pallet or a loaded pallet;

[0008] When the pallet status does not match the delivery task, the pallet abnormality information is output.

[0009] In one embodiment, identifying a pallet state corresponding to a pallet image includes: determining at least one candidate recognition region in the pallet image; for each candidate recognition region, extracting feature data of the candidate recognition region from the pallet image; determining a classification probability corresponding to the feature data, the classification probability including an empty pallet probability and a loaded pallet probability; and determining the pallet state corresponding to the pallet image based on the classification probability of each candidate recognition region.

[0010] In one embodiment, extracting feature data of a candidate recognition area from a pallet image includes: extracting multi-scale feature data of the candidate recognition area and context information of the candidate recognition area from the pallet image; and performing feature fusion on the multi-scale feature data and the context information to obtain feature data of the candidate recognition area.

[0011] In one embodiment, the conveying device includes a conveying device body, and a distance sensor and an image acquisition device respectively arranged on the conveying device body; controlling the conveying device to acquire a pallet image of the pallet to be transported includes: controlling the distance sensor to acquire a target distance between the conveying device and the pallet to be transported; and controlling the image acquisition device to acquire a pallet image of the pallet to be transported when the target distance is within a preset distance range.

[0012] In one embodiment, the method further includes: obtaining a target cigarette box image of a target cigarette box on the pallet to be transported when the pallet state matches the transport task and the transport task is a cargo pallet transport task; extracting target identification data from the target cigarette box image; and outputting cigarette box abnormality information when the target identification data is inconsistent with preset identification data.

[0013] In one embodiment, the pallet transport instruction also includes an unloading position; the method also includes: when the target identification data is consistent with the preset identification data, controlling the transport equipment to carry the pallet to be transported and the target cigarette box on the pallet to be transported; and controlling the transport equipment to move to the unloading position; when the unloading position is an idle position, controlling the transport equipment to place the pallet to be transported and the target cigarette box on the pallet to be transported at the unloading position.

[0014] In a second aspect, the present application also provides a safety detection device for pallet transportation, comprising:

[0015] A first control module is configured to control the transport device to capture a pallet image of a pallet to be transported in response to a pallet transport instruction; the pallet transport instruction includes a transport task, and the transport task includes an empty pallet transport task or a loaded pallet transport task;

[0016] A first recognition module is used to identify a tray state corresponding to the tray image, where the tray state includes an empty tray or a loaded tray;

[0017] The first output module is used to output pallet abnormality information when the pallet status does not match the transportation task.

[0018] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0019] In response to a pallet transport instruction, controlling the transport device to capture a pallet image of the pallet to be transported; the pallet transport instruction includes a transport task, and the transport task includes an empty pallet transport task or a loaded pallet transport task;

[0020] Identify the pallet status corresponding to the pallet image, where the pallet status includes an empty pallet or a loaded pallet;

[0021] When the pallet status does not match the delivery task, the pallet abnormality information is output.

[0022] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0023] In response to a pallet transport instruction, controlling the transport device to capture a pallet image of the pallet to be transported; the pallet transport instruction includes a transport task, and the transport task includes an empty pallet transport task or a loaded pallet transport task;

[0024] Identify the pallet status corresponding to the pallet image, where the pallet status includes an empty pallet or a loaded pallet;

[0025] When the pallet status does not match the delivery task, the pallet abnormality information is output.

[0026] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0027] In response to a pallet transport instruction, controlling the transport device to capture a pallet image of the pallet to be transported; the pallet transport instruction includes a transport task, and the transport task includes an empty pallet transport task or a loaded pallet transport task;

[0028] Identify the pallet status corresponding to the pallet image, where the pallet status includes an empty pallet or a loaded pallet;

[0029] When the pallet status does not match the delivery task, the pallet abnormality information is output.

[0030] The aforementioned pallet transport safety detection method, apparatus, device, medium, and program product, in response to pallet transport instructions, controls the transport equipment to capture images of the pallet to be transported, thereby providing a data foundation for subsequent abnormal pallet identification. By identifying the pallet status corresponding to the pallet image and outputting pallet abnormality information if the pallet status does not match the transport task, automated verification is achieved during the pallet transport process, preventing material transport anomalies due to incorrect pallet types and damage to the palletizer caused by mistakenly transporting a loaded pallet to an empty pallet stacker. This reduces safety hazards during transport and reduces manual verification costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1A A schematic structural diagram of a transport device in one embodiment;

[0033] Figure 1B is a schematic structural diagram of a second mounting frame in one embodiment;

[0034] Figure 1C Schematic diagram of the control principle of the main control unit in one embodiment;

[0035] Figure 2A A schematic flow chart of a safety detection method for pallet transportation according to one embodiment;

[0036] Figure 2B Schematic diagram of the process of constructing a pallet status recognition model in one embodiment;

[0037] Figure 2C Schematic diagram of the matching principle corresponding to the empty pallet transport task in one embodiment.

[0038] Figure 3 A flow chart of the steps for detecting a target cigarette box on a loading tray in one embodiment;

[0039] Figure 4 A schematic flow chart of a safety detection method for pallet transportation according to another embodiment;

[0040] Figure 5 A structural block diagram of a safety detection device for pallet transportation according to an embodiment;

[0041] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0043] The safety detection method for pallet transportation provided in the embodiment of the present application can be applied to Figure 1A The transport device shown in the figure may include an AGV (Automated Guided Vehicle) body, a distance sensor 1, an image acquisition device 2, a communication module 3 and a main control unit 4 installed on the AGV body.

[0044] Exemplarily, the AGV body includes a base and a mounting frame mounted on the base. The base has a loading surface for transporting materials. The mounting frame may include a first mounting frame 11 and a second mounting frame 12, with the first mounting frame 11 and the second mounting frame 12 forming a predetermined angle therebetween. The first mounting frame 11 is mounted on the base, and the second mounting frame 12 is connected to the first mounting frame 11. Optionally, the distance sensor 1 may be mounted on the first mounting frame 11, and the image acquisition device 2 may be mounted on the bottom of the second mounting frame 12. Optionally, the mounting frame may be a lifting structure for adjusting the height of the image acquisition device.

[0045] It should be noted that the present application does not impose any restrictions on the specific installation method and specific installation position of the distance sensor and the image acquisition device. For example, the image acquisition device can be installed on the first mounting bracket.

[0046] References Figure 1B Schematic diagram of the structure of the second mounting frame shown. At least one mounting hole 13 is provided on the second mounting frame 12 in the horizontal direction. Accordingly, the position of the image acquisition device can be adjusted and fixedly connected to the corresponding mounting hole. Exemplarily, the second mounting frame 12 may include a stainless steel bracket 121 and a steel sheet 122. Exemplarily, the stainless steel bracket 121 may be an isosceles triangle structure and may be fixed to the first mounting frame by magnetic attraction. The thickness of the steel sheet may be 3 cm to reduce the vibration of the stainless steel bracket 121.

[0047] Exemplarily, the distance sensor is used to measure the target distance between the AGV and a target material; the target material may include at least one of a pallet to be transported and a cigarette box to be transported. Optionally, the distance sensor may be a laser sensor. This application does not impose any restrictions on the specific sensor type of the distance sensor.

[0048] Illustratively, the image acquisition device is used to capture material images of materials. The material images may include at least one of an image of a pallet to be transported and an image of a target cigarette case. Optionally, the image acquisition device may include at least one of a camera and an industrial camera. This application does not impose any restrictions on the specific device type of the image acquisition device.

[0049] Exemplarily, the main control unit is connected to the communication module, the image acquisition device, and the distance sensor. This application does not impose any restrictions on the specific type and model of the main control unit. Optionally, the main control unit can be connected to a power module, which can be a 48V to 12V voltage conversion power supply circuit.

[0050] References Figure 1CThe figure shows a schematic diagram of the control principle of the main control unit. The main control unit obtains the target distance between the conveying equipment and the pallet to be transported, as measured by the distance sensor. When the target distance is within a preset range, the main control unit sends a capture command to the image acquisition device, causing it to capture an image of the pallet to be transported and send it to the main control unit. The AI ​​(artificial intelligence) module in the main control unit identifies the pallet status in the image. If the pallet status does not match the transport task, it sends a pallet anomaly message to the warehouse control system. Otherwise, the main control unit controls the conveying equipment to transport the pallet to be transported. The conveying equipment can include an AGV.

[0051] The main control unit may be connected to a power module for supplying power to the image acquisition device and the distance sensor.

[0052] The main control unit can establish a connection with the parameter server via a communication module. Exemplarily, the communication module can be a 5G communication module. The main control unit can send the pallet status to the parameter server via the communication module, so that the parameter server can match the pallet status with the system transport task and provide feedback on the matching result to the main control unit. Optionally, the pallet status can include an empty pallet or a loaded pallet. Optionally, the pallet status can also include material information corresponding to the pallet; accordingly, the system transport task can include information about the material to be transported.

[0053] In one embodiment, Figure 2A As shown, a safety detection method for pallet transportation is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understandable that the method can also be applied to a server, or to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0054] S210 , in response to a pallet transport instruction, controlling the transport device to collect a pallet image of the pallet to be transported; the pallet transport instruction includes a transport task, and the transport task includes an empty pallet transport task or a loaded pallet transport task.

[0055] The pallet to be transported can be understood as a pallet waiting to be transported by a transport device. The pallet to be transported can be an empty pallet or a loaded pallet. It can be understood that a loaded pallet is a pallet that carries materials, and an empty pallet is a pallet that does not carry materials.

[0056] In an optional embodiment, the pallet transport instruction may include a pickup location; accordingly, in response to the pallet transport instruction, the transport device may be controlled to move to the pickup location and to capture a pallet image of the pallet to be transported. For example, the pallet transport instruction may include a discharge location.

[0057] In an optional embodiment, the transport task may be determined based on the pickup location and / or unloading location corresponding to the pallet transport instruction. For example, if the unloading location is an empty pallet palletizer, the transport task may be determined to be an empty pallet transport task.

[0058] In another optional embodiment, the pickup location and / or unloading location may be determined to determine whether the transport task corresponding to the pallet transport instruction matches. If not, a notification message may be output.

[0059] In an optional embodiment, the conveying device includes a conveying device body, and a distance sensor and an image acquisition device respectively arranged on the conveying device body; accordingly, the distance sensor can be controlled to acquire the target distance between the conveying device and the pallet to be transported; when the target distance is within a preset distance range, the image acquisition device is controlled to acquire the pallet image of the pallet to be transported.

[0060] Optionally, the transport equipment body can be Figure 1A The AGV body shown in .

[0061] Optionally, the axis of the image acquisition device may be at a preset angle to the horizontal plane, which may be 45°, so that the image of the pallet to be transported is acquired at an inclination angle of 45°.

[0062] Optionally, the transport device body may be provided with a posture adjustment mechanism for adjusting the target posture of the image capture device; the target posture of the image capture device includes its position and posture. Accordingly, the posture adjustment mechanism can be controlled to adjust the posture of the image capture device, thereby controlling the image capture device to capture images of the pallet to be transported in different preset posture states.

[0063] S220: Identify a tray state corresponding to the tray image, where the tray state includes an empty tray or a loaded tray.

[0064] In an optional embodiment, at least one candidate recognition area in the pallet image can be determined; for each candidate recognition area, feature data of the candidate recognition area is extracted from the pallet image; a classification probability corresponding to the feature data is determined, the classification probability including an empty pallet probability and a loaded pallet probability; and based on the classification probability of each candidate recognition area, a pallet state corresponding to the pallet image is determined.

[0065] Optionally, before determining at least one candidate recognition region in the pallet image, the pallet image may be preprocessed. The preprocessing may include at least one of normalizing and resizing the pallet image.

[0066] Optionally, at least one candidate recognition region in the pallet image may be determined based on an RPN (Region Proposal Network).

[0067] Optionally, multi-scale feature data of the candidate recognition area and context information of the candidate recognition area are extracted from the tray image; and feature fusion is performed on the multi-scale feature data and the context information to obtain feature data of the candidate recognition area.

[0068] For example, the multi-scale feature data of the candidate recognition region and the context information of the candidate recognition region can be extracted in a feature pyramid network.

[0069] Optionally, overlapping, redundant candidate recognition regions can be removed using non-maximum suppression, and the pallet state corresponding to the pallet image can be determined based on the classification probability of the candidate recognition region with the highest confidence. For example, for each candidate recognition region, the confidence level can be determined based on the target presence probability and classification probability corresponding to the candidate recognition region. The target presence probability can be understood as the probability that a pallet exists in the candidate recognition region.

[0070] In an optional embodiment, the tray state corresponding to the tray image may be identified by an AI (Artificial Intelligence) model or a trained tray state recognition model.

[0071] refer to Figure 2B The figure shows a schematic diagram of the construction process of the pallet state recognition model. The construction process of the pallet state recognition model may include: establishing a neural network structure and setting initial values ​​for the neural network parameters; extracting feature data of the training pallet image for each training pallet image sample; generating candidate recognition regions that may contain an object based on a region proposal network, where the object can be a pallet; inputting the feature data of the training pallet image into the constructed feature pyramid network, fusing feature maps from different levels to enhance the model's perception of multi-scale objects, extracting multi-scale feature data and contextual information of the candidate recognition regions from the feature maps from different levels; concatenating and fusing the multi-scale feature data and contextual information to form the final feature representation; outputting the classification results of the candidate recognition regions through a fully connected layer, and updating the network parameters of the pallet state recognition model based on the classification results and the labels of the training pallet images. These steps are repeated until the pallet state recognition model converges.

[0072] S230: When the pallet status does not match the delivery task, output pallet abnormality information.

[0073] In an optional embodiment, if the tray status is an empty tray and the transport task is a loaded tray transport task, the tray status and the transport task do not match, and a tray abnormality message is output. If the tray status is a loaded tray and the transport task is an empty tray transport task, the tray status and the transport task do not match, and a tray abnormality message is output.

[0074] For ease of understanding, the target delivery task is an empty pallet delivery task. Figure 2C The figure shows the matching principle for an empty pallet delivery task. When the pallet is in place, that is, within the preset distance threshold, a 45-degree angled camera captures the image, and the visual AI module determines whether it is an empty pallet or a loaded pallet. If the output is empty, the delivery task continues; otherwise, an alarm is issued and manual processing is performed.

[0075] In an optional embodiment, the pallet transport instruction may include an unloading location. Accordingly, when the pallet status matches the transport task, the transport device is controlled to load the pallet to be transported and transport it to the unloading location.

[0076] In one optional embodiment, outputting the pallet exception information may include sending the pallet exception information to a warehouse management system terminal. The pallet exception information may include at least one of the pallet location, pallet transport instructions, and transport equipment identification. In another optional embodiment, outputting the pallet exception information may also include stopping the transport equipment to retrieve the goods.

[0077] The above-mentioned pallet transport safety detection method controls the transport equipment to capture images of the pallet to be transported in response to pallet transport instructions, thus providing a data foundation for subsequent abnormal pallet identification. By identifying the pallet status corresponding to the pallet image and outputting pallet abnormality information if the pallet status does not match the transport task, automated verification is achieved during the pallet transport process, preventing material transport anomalies caused by incorrect pallet type and damage to the palletizer caused by mistakenly transporting a loaded pallet to an empty pallet stacker. This reduces safety hazards during transport and reduces manual verification costs.

[0078] In an optional embodiment, the above method can be applied to a service system, and the edge deployment of the neural network algorithm and the use of the YOLOv5 object detection algorithm to identify the pallet state corresponding to the pallet image can be implemented on the service platform. Before using the service platform to deploy the YOLOv5 model, it is first necessary to convert the trained YOLOv5 model (for example, in ONNX format) to a format supported by the service platform (for example, in rknn format). This conversion process can be completed using service tools, including steps such as model parsing, optimization, and conversion. For example, the YOLOv5 model can be deployed on an embedded device and executed by an inference engine to perform real-time object detection on the input image.

[0079] Based on the technical solutions of the above embodiments, the present application also provides an optional embodiment, in which a detection step of a target cigarette box on the loading tray is added.

[0080] refer to Figure 3 The following is a flow chart of the steps for detecting a target cigarette box on a carrier tray, including:

[0081] S310 : When the pallet status matches the transport task and the transport task is a cargo pallet transport task, a target cigarette box image of the target cigarette box on the pallet to be transported is acquired.

[0082] In an optional embodiment, the acquisition distance between the image acquisition device and the target cigarette box may be acquired, and when the acquisition process is within a preset acquisition range, the image acquisition device is controlled to acquire the target cigarette box image on the target cigarette box.

[0083] In another optional embodiment, the image area where the target cigarette box is located may be identified from the tray image to obtain a target cigarette box image of the target cigarette box.

[0084] S320: Extract target identification data from the target cigarette box image.

[0085] In an optional embodiment, the target cigarette box image can be input into a pre-trained identification recognition model to obtain target identification data; the target cigarette box image can be binarized to obtain a binary image; the binarized image can be segmented into characters to obtain character segmentation data; and the character segmentation data can be subjected to character recognition to obtain target identification data.

[0086] S330: When the target identification data is inconsistent with the preset identification data, output abnormal information of the cigarette box.

[0087] In an optional embodiment, the pallet transport instruction may include preset identification data.

[0088] In an optional embodiment, when the target identification data is consistent with the preset identification data, the conveying equipment can be controlled to carry the pallet to be transported and the target cigarette box on the pallet to be transported; and the conveying equipment can be controlled to move to the unloading position; when the unloading position is an idle position, the conveying equipment can be controlled to place the pallet to be transported and the target cigarette box on the pallet to be transported at the unloading position.

[0089] In the above steps, when the pallet status matches the delivery task and the delivery task is a pallet-loaded delivery task, a target cigarette case image of the target cigarette case on the pallet to be delivered is obtained, thereby providing a data basis for subsequently determining whether the target cigarette case matches the cigarette case to be delivered. By extracting the target identification data from the target cigarette case image and outputting cigarette case abnormality information if the target identification data does not match the preset identification data, this prevents target cigarette cases that do not match the actual delivery task from being delivered to the unloading location, thereby affecting normal production progress and helping to reduce manual review costs.

[0090] Based on the above embodiments, a safety detection method for pallet transportation is described in detail.

[0091] refer to Figure 4 In another embodiment, a safety detection method for pallet transportation includes:

[0092] S410 , in response to a pallet transport instruction, controlling a distance sensor to collect a target distance between the transport device and the pallet to be transported.

[0093] The pallet transport instruction includes a transport task, and the transport task includes an empty pallet transport task or a loaded pallet transport task.

[0094] S420: When the target distance is within a preset distance range, control the image acquisition device to acquire an image of the pallet to be transported.

[0095] S430: Determine at least one candidate recognition area in the tray image.

[0096] S440 : For each candidate recognition region, extract multi-scale feature data of the candidate recognition region and context information of the candidate recognition region from the tray image.

[0097] S450 , performing feature fusion on the multi-scale feature data and the context information to obtain feature data of the candidate recognition area.

[0098] S460: Determine the classification probability corresponding to the feature data, where the classification probability includes an empty tray probability and a loaded tray probability.

[0099] S470 , determining a tray state corresponding to the tray image based on the classification probability of each candidate recognition area; the tray state includes an empty tray or a loaded tray.

[0100] S480: When the pallet status does not match the delivery task, output pallet abnormality information.

[0101] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0102] Based on the same inventive concept, embodiments of the present application also provide a pallet transport safety detection device for implementing the aforementioned pallet transport safety detection method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more pallet transport safety detection device embodiments provided below can be found in the aforementioned limitations of the pallet transport safety detection method and will not be further elaborated here.

[0103] In an exemplary embodiment, Figure 5 As shown, a safety detection device for pallet transportation is provided, comprising: a first control module 510, a first identification module 520 and a first output module 530, wherein:

[0104] The first control module 510 is configured to control the transport device to capture a pallet image of a pallet to be transported in response to a pallet transport instruction; the pallet transport instruction includes a transport task, and the transport task includes an empty pallet transport task or a loaded pallet transport task;

[0105] A first recognition module 520 is used to identify a tray state corresponding to the tray image, where the tray state includes an empty tray or a loaded tray;

[0106] The first output module 530 is configured to output pallet abnormality information when the pallet status does not match the transport task.

[0107] In one embodiment, the first recognition module 520 includes: a first determination unit for determining at least one candidate recognition area in the pallet image; a first extraction unit for extracting feature data of the candidate recognition area from the pallet image for each candidate recognition area; a second determination unit for determining the classification probability corresponding to the feature data, the classification probability including the probability of an empty pallet and the probability of a loaded pallet; and a third determination unit for determining the pallet state corresponding to the pallet image based on the classification probability of each candidate recognition area.

[0108] In one embodiment, the first extraction unit includes: a first extraction subunit, configured to extract multi-scale feature data of a candidate recognition area and context information of the candidate recognition area from the pallet image; and a fusion subunit, configured to perform feature fusion on the multi-scale feature data and the context information to obtain feature data of the candidate recognition area.

[0109] In one embodiment, the conveying device includes a conveying device body, and a distance sensor and an image acquisition device respectively arranged on the conveying device body; the first control module 510 includes: a first control unit, used to control the distance sensor to acquire the target distance between the conveying device and the pallet to be transported; a second control unit, used to control the image acquisition device to acquire the pallet image of the pallet to be transported when the target distance is within a preset distance range.

[0110] In one embodiment, the system further includes: a first acquisition module, configured to acquire a target cigarette box image of a target cigarette box on a pallet to be transported when the pallet state matches the transport task and the transport task is a cargo pallet transport task; a first extraction module, configured to extract target identification data from the target cigarette box image; and a second output module, configured to output cigarette box abnormality information when the target identification data is inconsistent with preset identification data.

[0111] In one embodiment, the pallet transport instruction also includes an unloading position, and also includes: a second control module, used to control the transport equipment to carry the pallet to be transported and the target cigarette box on the pallet to be transported when the target identification data is consistent with the preset identification data; a third control module, used to control the transport equipment to move to the unloading position; and a fourth control module, used to control the transport equipment to place the pallet to be transported and the target cigarette box on the pallet to be transported at the unloading position when the unloading position is an idle position.

[0112] Each module in the aforementioned pallet transport safety detection device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor within a computer device in hardware form, or stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0113] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means, and the wireless means can be implemented via Wi-Fi, a mobile cellular network, near-field communication (NFC), or other technologies. When executed by the processor, the computer program implements a safety detection method for pallet transportation. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0114] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0115] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0116] In response to a pallet transport instruction, controlling the transport device to capture a pallet image of the pallet to be transported; the pallet transport instruction includes a transport task, and the transport task includes an empty pallet transport task or a loaded pallet transport task;

[0117] Identify the pallet status corresponding to the pallet image, where the pallet status includes an empty pallet or a loaded pallet;

[0118] When the pallet status does not match the delivery task, the pallet abnormality information is output.

[0119] In one embodiment, when the processor executes the computer program, it further implements the following steps: determining at least one candidate identification area in the pallet image; for each candidate identification area, extracting feature data of the candidate identification area from the pallet image; determining a classification probability corresponding to the feature data, the classification probability including an empty pallet probability and a loaded pallet probability; and determining a pallet state corresponding to the pallet image based on the classification probability of each candidate identification area.

[0120] In one embodiment, when the processor executes the computer program, it further implements the following steps: extracting multi-scale feature data of the candidate recognition area and context information of the candidate recognition area from the pallet image; and performing feature fusion on the multi-scale feature data and the context information to obtain feature data of the candidate recognition area.

[0121] In one embodiment, the conveying device includes a conveying device body, and a distance sensor and an image acquisition device respectively arranged on the conveying device body; when the processor executes the computer program, it also implements the following steps: controlling the distance sensor to acquire the target distance between the conveying device and the pallet to be transported; when the target distance is within a preset distance range, controlling the image acquisition device to acquire the pallet image of the pallet to be transported.

[0122] In one embodiment, when the processor executes the computer program, the following steps are further implemented: when the pallet status matches the transport task and the transport task is a cargo pallet transport task, a target cigarette box image of the target cigarette box on the pallet to be transported is obtained; target identification data in the target cigarette box image is extracted; and when the target identification data is inconsistent with the preset identification data, cigarette box abnormality information is output.

[0123] In one embodiment, the pallet transport instruction also includes an unloading position; when the processor executes the computer program, it also implements the following steps: when the target identification data is consistent with the preset identification data, controlling the transport equipment to carry the pallet to be transported and the target cigarette box on the pallet to be transported; and controlling the transport equipment to move to the unloading position; when the unloading position is an idle position, controlling the transport equipment to place the pallet to be transported and the target cigarette box on the pallet to be transported at the unloading position.

[0124] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0125] In response to a pallet transport instruction, controlling the transport device to capture a pallet image of the pallet to be transported; the pallet transport instruction includes a transport task, and the transport task includes an empty pallet transport task or a loaded pallet transport task;

[0126] Identify the pallet status corresponding to the pallet image, where the pallet status includes an empty pallet or a loaded pallet;

[0127] When the pallet status does not match the delivery task, the pallet abnormality information is output.

[0128] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: determining at least one candidate recognition area in the pallet image; for each candidate recognition area, extracting feature data of the candidate recognition area from the pallet image; determining a classification probability corresponding to the feature data, the classification probability including an empty pallet probability and a loaded pallet probability; and determining a pallet state corresponding to the pallet image based on the classification probability of each candidate recognition area.

[0129] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: extracting multi-scale feature data of the candidate recognition area and context information of the candidate recognition area from the pallet image; and performing feature fusion on the multi-scale feature data and the context information to obtain feature data of the candidate recognition area.

[0130] In one embodiment, the conveying device includes a conveying device body, and a distance sensor and an image acquisition device respectively arranged on the conveying device body; when the computer program is executed by the processor, the following steps are also implemented: controlling the distance sensor to acquire the target distance between the conveying device and the pallet to be transported; when the target distance is within a preset distance range, controlling the image acquisition device to acquire the pallet image of the pallet to be transported.

[0131] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: when the pallet status matches the transport task and the transport task is a cargo pallet transport task, a target cigarette box image of the target cigarette box on the pallet to be transported is obtained; target identification data in the target cigarette box image is extracted; and when the target identification data is inconsistent with the preset identification data, cigarette box abnormality information is output.

[0132] In one embodiment, the pallet transport instruction also includes an unloading position; when the computer program is executed by the processor, the following steps are also implemented: when the target identification data is consistent with the preset identification data, the transport device is controlled to carry the pallet to be transported and the target cigarette box on the pallet to be transported; and, the transport device is controlled to move to the unloading position; when the unloading position is an idle position, the transport device is controlled to place the pallet to be transported and the target cigarette box on the pallet to be transported at the unloading position.

[0133] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0134] In response to a pallet transport instruction, controlling the transport device to capture a pallet image of the pallet to be transported; the pallet transport instruction includes a transport task, and the transport task includes an empty pallet transport task or a loaded pallet transport task;

[0135] Identify the pallet status corresponding to the pallet image, where the pallet status includes an empty pallet or a loaded pallet;

[0136] When the pallet status does not match the delivery task, the pallet abnormality information is output.

[0137] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: determining at least one candidate recognition area in the pallet image; for each candidate recognition area, extracting feature data of the candidate recognition area from the pallet image; determining a classification probability corresponding to the feature data, the classification probability including an empty pallet probability and a loaded pallet probability; and determining a pallet state corresponding to the pallet image based on the classification probability of each candidate recognition area.

[0138] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: extracting multi-scale feature data of the candidate recognition area and context information of the candidate recognition area from the pallet image; and performing feature fusion on the multi-scale feature data and the context information to obtain feature data of the candidate recognition area.

[0139] In one embodiment, the conveying device includes a conveying device body, and a distance sensor and an image acquisition device respectively arranged on the conveying device body; when the computer program is executed by the processor, the following steps are also implemented: controlling the distance sensor to acquire the target distance between the conveying device and the pallet to be transported; when the target distance is within a preset distance range, controlling the image acquisition device to acquire the pallet image of the pallet to be transported.

[0140] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: when the pallet status matches the transport task and the transport task is a cargo pallet transport task, a target cigarette box image of the target cigarette box on the pallet to be transported is obtained; target identification data in the target cigarette box image is extracted; and when the target identification data is inconsistent with the preset identification data, cigarette box abnormality information is output.

[0141] In one embodiment, the pallet transport instruction also includes an unloading position; when the computer program is executed by the processor, the following steps are also implemented: when the target identification data is consistent with the preset identification data, the transport device is controlled to carry the pallet to be transported and the target cigarette box on the pallet to be transported; and, the transport device is controlled to move to the unloading position; when the unloading position is an idle position, the transport device is controlled to place the pallet to be transported and the target cigarette box on the pallet to be transported at the unloading position.

[0142] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0143] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0144] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A safety detection method for pallet transportation, characterized in that: include: In response to a pallet transport instruction, controlling the transport device to capture a pallet image of the pallet to be transported; the pallet transport instruction includes a transport task, and the transport task includes an empty pallet transport task or a loaded pallet transport task; Identifying a tray state corresponding to the tray image, where the tray state includes an empty tray or a loaded tray; When the pallet status does not match the transport task, pallet abnormality information is output.

2. The method according to claim 1, characterized in that The identifying the tray state corresponding to the tray image includes: determining at least one candidate recognition region in the pallet image; For each candidate identification area, extracting feature data of the candidate identification area from the tray image; Determining a classification probability corresponding to the feature data, the classification probability including an empty tray probability and a loaded tray probability; The tray state corresponding to the tray image is determined according to the classification probability of each candidate recognition area.

3. The method according to claim 2, characterized in that The step of extracting feature data of the candidate recognition area from the tray image includes: extracting multi-scale feature data of the candidate recognition area and context information of the candidate recognition area from the pallet image; The multi-scale feature data and the context information are subjected to feature fusion to obtain feature data of the candidate recognition area.

4. The method according to any one of claims 1 to 3, characterized in that The transport device includes a transport device body, and a distance sensor and an image acquisition device respectively arranged on the transport device body; The controlling the transport device to collect the pallet image of the pallet to be transported includes: Controlling the distance sensor to collect the target distance between the transport device and the pallet to be transported; When the target distance is within a preset distance range, the image acquisition device is controlled to acquire a pallet image of the pallet to be transported.

5. The method according to any one of claims 1 to 3, characterized in that Also includes: When the pallet state matches the transport task and the transport task is a cargo pallet transport task, acquiring a target cigarette box image of the target cigarette box on the pallet to be transported; extracting target identification data from the target cigarette box image; When the target identification data is inconsistent with the preset identification data, the cigarette box abnormality information is output.

6. The method according to claim 5, characterized in that The pallet delivery instruction also includes a unloading location; the method further includes: When the target identification data is consistent with the preset identification data, controlling the transport device to carry the pallet to be transported and the target cigarette box on the pallet to be transported; and Controlling the transport device to move to the unloading position; When the unloading position is an idle position, the transport device is controlled to place the pallet to be transported and the target cigarette box on the pallet to be transported at the unloading position.

7. A safety detection device for pallet transportation, characterized in that: The device comprises: A first control module is configured to control the transport device to capture a pallet image of a pallet to be transported in response to a pallet transport instruction; the pallet transport instruction includes a transport task, and the transport task includes an empty pallet transport task or a loaded pallet transport task; A first recognition module is used to identify a tray state corresponding to the tray image, where the tray state includes an empty tray or a loaded tray; The first output module is configured to output pallet abnormality information when the pallet status does not match the transport task.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images