Body detection-based delivery system management
By using image cameras and thermal cameras combined with machine learning models in the conveyor system, physical intrusions around the conveyor can be automatically detected, solving the problem of reliance on manual monitoring in the conveyor system, improving safety and security levels, and reducing labor costs.
Patent Information
- Application Number
- CN202580001217.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-17
- Publication Date
- 2026-11-17
AI Technical Summary
In existing object conveying systems, accidental entry of people or animals into prohibited areas of the conveyor belt can cause physical injury and equipment damage. Existing solutions rely heavily on manual monitoring and lack automated intrusion detection capabilities.
The system uses image cameras and thermal cameras combined with machine learning models to collect image data and thermal data respectively. The machine learning model detects physical intrusions around the conveyor and manages the conveyor system based on the detection results, including suspending the conveyor and providing alarms.
It enables automated intrusion detection of the transmission system, improving security and protection levels, reducing labor costs, and decreasing reliance on manual monitoring.
Smart Images

Figure CN123125626A_ABST
Abstract
Description
Technical Field
[0001] This disclosure generally relates to computer technology, and more specifically, to methods, apparatus, and computer program products for managing object delivery systems based on body detection. Background Technology
[0002] Currently, object conveyor systems are widely used in various fields. For example, conveyor belts can be used to move passenger luggage in transportation systems (such as airports), and they can also be used to deliver packages in parcel delivery systems. During the operation of a conveyor belt, people (such as adults or children) or animals may accidentally enter prohibited areas of the conveyor system, resulting in bodily injury and / or damage to the conveyor belt. While video surveillance systems can assist in human intrusion detection to some extent, existing solutions heavily rely on manual monitoring. Therefore, it is desirable to automatically implement human intrusion detection to ensure the proper operation of object conveyor systems. Summary of the Invention
[0003] In a first aspect of this disclosure, a method for managing an object transport system based on body detection is provided. The method includes: receiving image data from an image camera and thermal data from a thermal camera, respectively, within a predetermined area surrounding a transport device in the object transport system; detecting bodies within the predetermined area using a machine learning model based on the image data and thermal data; and managing the object transport system based on the detected bodies.
[0004] In a second aspect of this disclosure, an electronic device is provided. The electronic device includes a computer processor coupled to a computer-readable storage unit, the storage unit including instructions that, when executed by the computer processor, implement the method according to the first aspect of this disclosure.
[0005] In a third aspect of this disclosure, a non-transitory computer program product is provided, the computer program product including a computer-readable storage medium having program instructions embodied thereon, the program instructions being executable by an electronic device to cause the electronic device to perform the method according to the first aspect of this disclosure.
[0006] The present invention is provided to introduce a series of concepts in a simplified form, which will be further described in the detailed description below. The present invention is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description
[0007] The features, advantages, and other aspects of the embodiments of this disclosure will become more apparent from the more detailed description in the accompanying drawings. Several implementations of this disclosure are shown schematically and are not intended to limit the scope of this disclosure. In the accompanying drawings:
[0008] Figure 1 A block diagram of an example object transport system in which implementations of this disclosure may be carried out is schematically shown;
[0009] Figure 2 A block diagram illustrating an implementation of the present disclosure for managing object delivery based on body detection is shown.
[0010] Figure 3 An example process for detecting body regions based on candidate image regions and candidate hot regions, according to an implementation of this disclosure, is illustrated.
[0011] Figure 4 The illustrations show the results of body detection under various conditions according to the implementation of this disclosure;
[0012] Figure 5 The illustration schematically shows the results of a body detection according to an implementation of this disclosure;
[0013] Figure 6 An example interface of the management system for an object transfer system according to an implementation of this disclosure is illustrated.
[0014] Figure 7 The illustration schematically depicts an object transport environment according to an implementation of this disclosure;
[0015] Figure 8 The flowchart schematically illustrates a method for managing an object delivery system based on body detection, according to an implementation of this disclosure; and
[0016] Figure 9 A block diagram of a computing device in which various implementations of the present disclosure may be implemented is shown schematically. Detailed Implementation
[0017] The principles of this disclosure will now be described with reference to some implementations. It should be understood that these implementations are described for illustrative purposes only and to assist those skilled in the art in understanding and implementing this disclosure, and do not imply any limitation on the scope of this disclosure. The disclosure described herein can be implemented in various ways other than those described below.
[0018] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.
[0019] References to "an implementation," "implementation," "example implementation," etc., in this disclosure indicate that the described implementation may include a specific feature, structure, or characteristic, but not every implementation necessarily includes that specific feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same implementation. Additionally, when a specific feature, structure, or characteristic is described in conjunction with an example implementation, it is assumed that the combination of other implementations affects that feature, structure, or characteristic in a way that is within the knowledge of someone skilled in the art, whether explicitly stated or not.
[0020] It should be understood that although the terms “first” and “second” may be used in this document to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the example implementation. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.
[0021] The terminology used herein is for the purpose of describing a particular implementation only and is not intended to limit the example implementations. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that the terms “comprising,” “including,” and “having” as used herein specify the presence of the stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.
[0022] The principles of this disclosure will now be described with reference to some implementations. It should be understood that these implementations are described for illustrative purposes only and to assist those skilled in the art in understanding and implementing this disclosure, and do not imply any limitation on the scope of this disclosure. The disclosure described herein can be implemented in various ways other than those described below. In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.
[0023] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of relevant laws, regulations and rules.
[0024] It is understood that before using the technical solutions disclosed in the various implementations of this disclosure, users should be notified in an appropriate manner, in accordance with relevant laws and regulations, of the types, scope of use, and usage scenarios of the personal information involved in this disclosure, and user authorization should be obtained.
[0025] For example, in response to receiving an active request from a user, a prompt message is sent to the user to explicitly notify the user that the requested operation will require the acquisition and use of the user's personal information. Therefore, the user can independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media that perform the operations of the technical solutions disclosed herein.
[0026] As an optional but not limited implementation, the method of sending a prompt message to the user in response to an active request may include, for example, a pop-up window, in which the prompt message may be presented in text form. Furthermore, the pop-up window may also contain a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0027] It is understood that the above notification and user authorization process are merely illustrative and do not limit the implementation of this disclosure. Other methods that comply with applicable laws and regulations are also applicable to the implementation of this disclosure.
[0028] As mentioned above, during the operation of the conveyor system, a person or animal may accidentally enter a prohibited area, resulting in bodily injury and / or damage to the conveyor belt. For ease of illustration, a body detection environment can be described using a passenger transport environment as an example; for instance, the conveyor system may include a conveyor belt at a baggage claim area in an airport. Figure 1 A block diagram 100 schematically illustrates an object transfer system 110 that can implement the embodiments of this disclosure.
[0029] like Figure 1 As shown, the object conveyor system can be a baggage handling system equipped in the arrival hall of an airport. For example, conveyor 120 can be deployed in the baggage claim area of the airport, and bags 130 and 132 can be delivered to passengers. Passengers 120 and 122 can stand near conveyor 120 and wait for their baggage. However, sometimes children and / or adults climb onto conveyor 120 and move with it, which may pose security and / or safety risks. While video surveillance systems can assist in human intrusion detection to some extent, existing solutions rely heavily on manual monitoring. Typically, there are dozens of conveyors in baggage claim areas and self-service baggage drop-off areas, and monitoring each conveyor may result in an unacceptable workload. Therefore, it is desirable to automate human intrusion detection to ensure the proper operation of the object conveyor system.
[0030] In view of this, this disclosure proposes a solution for managing object transport systems based on body detection. Generally, an image camera and a thermal camera are deployed near the transporter to acquire image data and thermal data, respectively. A machine learning model is then used to detect bodies based on the acquired image and thermal data. See below for more details. Figure 2 , Figure 2 A block diagram 200 for managing object transfer based on body detection is shown schematically according to an implementation of the present disclosure.
[0031] like Figure 2 As shown, image camera 210 and thermal camera 220 can be deployed within a predetermined area around the conveying equipment (such as conveyor 120) in object conveying system 250. Image data 212 is received from image camera 210 and thermal data 222 is received from thermal camera 220, respectively. The image data 212 and thermal data 222 can then be input into machine learning model 230, which can detect bodies 240 within the predetermined area based on the image data 212 and thermal data 222. Furthermore, object conveying system 250 can be managed based on the detected bodies 240.
[0032] Utilizing these implementation methods, the proposed solution can automatically detect body intrusions through machine learning models. Since thermal cameras can sense heat from people and animals, body detection can be achieved more accurately and effectively based on both image and thermal data, thereby improving security levels in airports and reducing labor costs. Furthermore, this human intrusion detection solution leverages cutting-edge video analytics tools powered by advanced AI (artificial intelligence) deep learning algorithms to intelligently and accurately detect the presence of humans on baggage carousels, preventing any unauthorized intrusions into restricted areas via moving baggage carousels.
[0033] In an implementation of this disclosure, to detect a body within a predetermined area based on image data and thermal data, at least one body region can be identified based on the image data and thermal data; and then, in response to determining that the at least one body region belongs to the same body, a body is determined based on the at least one body region. Here, thermal data 222 can utilize different colors to represent the temperature distribution in the environment, typically transitioning from cool tones (such as blue) to warm tones (such as red) to visually display temperature changes. Typically, due to various thermal noises in the environment, the surface temperature of the human body is approximately in the temperature range between 30 and 37°C, so regions in the thermal data corresponding to the aforementioned temperature range can be identified for body detection. For example, thermal data 222 can be converted into a thermal image with the same resolution (e.g., width * height) as image data 212, and then superimposed on image data 212 for body detection.
[0034] Using these implementations, both image data and thermal data are considered in body detection, so thermal data can be used to remove image regions that have a human appearance but have a low temperature.
[0035] In an implementation of this disclosure, to identify at least one body region from image data and thermal data, a machine learning model can identify at least one candidate image region associated with the body from the image data, and simultaneously, the machine learning model can identify at least one candidate thermal region associated with the body from the thermal data. The machine learning model can then determine at least one body region based on the at least one candidate image region and the at least one candidate thermal region.
[0036] Specifically, the machine learning model may include: an image processing module for identifying one or more image regions related to the body; and a heat processing module for identifying one or more heat regions related to the body. Image data 212 can be input into the image processing module, and at least one candidate image region can be output. Simultaneously, heat data 222 can be input into the heat processing module, and at least one candidate heat region can be output.
[0037] In these implementations, the image processing module and the heat processing module can be trained based on real samples. For example, images and (one or more) labeled body regions, along with (one or more) classifications of the labeled body regions, can be used as training samples. Images can be input into the image processing module to determine predictions of image regions. The predictions can be compared with the labeled body regions to determine a loss, and the image processing model can then be trained in a direction that minimizes the loss. At this point, the image processing module can identify body-related image regions and also provide classifications of the image regions (e.g., head, arm, etc.). Furthermore, the heat processing module can be trained similarly. Alternatively and / or additionally, regions whose color falls within a predetermined range (corresponding to human body temperature) can be extracted as heat regions.
[0038] For more detailed information, please refer to [link / reference]. Figure 3 , Figure 3 An example process 300 for detecting body regions based on candidate image regions and candidate hot regions, according to an implementation of this disclosure, is illustrated schematically. Figure 3As shown, the image processing module can output candidate image region 310, and the heat processing module can output candidate hot region 320. Furthermore, candidate image region 310 and candidate hot region 320 can be compared to determine whether candidate image region 310 actually involves a part of the body. For example, as shown by arrow 330, the overlapping area between candidate image region 310 and candidate hot region 320 can be determined. The confidence level can be obtained by calculating the Intersection Over Union (IOU) between candidate image region 310 and candidate hot region 320. IOU can be calculated as follows:
[0039] In formula (1), box image This indicates the position of candidate image region 310 in image data 212, and box thermal This indicates the location of candidate hot region 320 in hot data 222. A threshold IOU (e.g., 50% or another value) can be defined to determine whether candidate image region 310 actually involves a part of the body. If the IOU determined according to formula (1) above is equal to or greater than the threshold IOU, then candidate image region 310 is considered a body region. Through these implementations, body regions can be determined in a more efficient and accurate manner.
[0040] To increase the accuracy of body detection, body regions should be examined to determine whether they belong to the same body. Specifically, in response to the distribution of at least one body region satisfying a predetermined body pose, it is determined that the at least one body region belongs to the same body. Machine learning models can include a pose checking module to achieve this functionality. For example, if one region corresponds to a person's head and another region corresponds to a person's arm, but the distance between these two regions is greater than a reasonable value (e.g., greater than 2 meters), the pose checking module can output a negative result indicating that the two body regions do not belong to the same body. The pose checking module can be pre-trained, for example, using positive and negative training samples to train the pose checking model, such that the module can output a probability level indicating whether the input body regions belong to the same body.
[0041] To further improve the accuracy of body detection, the total area of at least one body region should be greater than a predetermined percentage of the body area. For example, this predetermined percentage could be set to, say, 20%, 30%, or another value. In another example, regions corresponding to different parts of the body can have different weights. Specifically, the head can have a larger weight, and the legs can have a smaller weight. Using these implementations, the accuracy of body detection can be further increased.
[0042] In the implementation of this disclosure, image data consists of image frames from an image sequence (such as an image video) captured by an image camera, and thermal data consists of hot frames from a thermal sequence (such as a thermal video) captured by a thermal camera. The body is detected in additional image frames within the image sequence and additional hot frames within the thermal sequence. Specifically, the image camera and the thermal camera can capture separate videos, and a machine learning model can be used to continuously analyze the videos. If the duration of a body appearing in the video exceeds a threshold duration (e.g., 1 second or 2 seconds), the machine learning model can output a positive result for body detection. Using these implementations, errors caused by detection in a single frame can be mitigated, and the accuracy level can be further increased.
[0043] Sometimes, ambient temperature affects the accuracy of human detection derived from thermal data. For example, when the ambient temperature is close to or even higher than body temperature, the body cannot be correctly identified from the thermal data due to the similarity between human body temperature and ambient temperature. In such cases, body detection should rely more heavily on image data.
[0044] In the implementation of this disclosure, in order to detect a body within a predetermined area based on image data and thermal data, the ambient temperature of the object transport system can be obtained; image weights for the image data can be determined, and thermal weights for the thermal data can be determined; then the body can be determined based on the image weights, image data, thermal weights, and thermal data.
[0045] In these implementations, the machine learning model can also include a weighting module for receiving ambient temperature and detecting the body based on the ambient temperature, thermal data, and image data. For example, if the ambient temperature is close to or even higher than normal body temperature, the thermal weight can be reduced, and thus the relative importance of the image data can be increased. Therefore, errors caused by high ambient temperatures can be mitigated, and thus the accuracy of body detection can be increased. The weighting module can be trained to output appropriate weights for the image and thermal data.
[0046] In embodiments of this disclosure, modules in the machine learning model can be pre-trained individually. Alternatively and / or additionally, these modules can be fine-tuned together. For example, pre-trained modules can be trained together using training samples. Training samples may include ambient temperature, image data, and thermal data with ground truth labels. A loss can be determined between the body's predictions and the ground truth labels, and modules can be updated in a direction that minimizes the loss.
[0047] Figure 4 The illustration schematically shows the results 400 of body detection under various conditions according to an implementation of this disclosure. For example... Figure 4As shown, the machine learning model can detect the entire body and parts of the body. Furthermore, because ambient temperature is taken into account in body detection, the machine learning model can also detect the body even in the presence of nearby thermal noise.
[0048] In the implementation disclosed herein, the body can be labeled in the image data; see details below. Figure 5 . Figure 5 The illustration schematically shows the result 500 of a body detection according to an implementation of this disclosure. For example... Figure 5 As shown, image 510 can be input into the machine learning model, and no body is detected in image 510. Furthermore, regarding image 520, a child is behind a suitcase, with their head, one hand, and one foot exposed. In this case, the machine learning model can provide a box 722 to indicate the child's head, a box 724 to indicate the child's foot, and a box 726 to indicate the child's hand. The body can then be clearly indicated in the image, and the administrator of the delivery system can be notified to take appropriate action. In image 530, a ball is placed on the suitcase. Although the ball's shape resembles a head, no human body is detected from the thermal data, therefore no one is detected in image 530. Using these implementations, even if a human mask is placed on the suitcase, the model will not mistakenly identify the mask as a face due to the mask's low temperature.
[0049] Typically, passengers wait around the conveyor belt for their luggage. While a passenger's body is detected, the passenger is standing within the safe area around the conveyor belt and should not be considered a human intruder. In an implementation of this disclosure, to manage the object transport system based on body movement, the relationship between the body and the conveyor equipment can be detected; and the object transport system can be managed in response to determining that the relationship meets predetermined risk criteria. Here, the machine learning model may also include a risk module for detecting whether the detected body is in a risky situation. Risky situations can include various types, such as the body standing on the conveyor belt, the body sitting on the conveyor belt, etc. The risk module can be trained using positive and negative training samples. Once trained, the risk module can have the ability to identify risky situations from normal conditions. Therefore, the machine learning model can provide an alert only when a hazard is detected.
[0050] In the implementation of this disclosure, the machine learning model can analyze passenger movement based on continuously acquired image and thermal data. For example, when a passenger is waiting near a conveyor belt and / or retrieving luggage from it, the machine learning model can accurately identify his / her movements. The model can detect whether a passenger's posture is in a dangerous state. For example, if a passenger falls onto the conveyor belt while retrieving luggage, physical intrusion can be detected and appropriate measures can be triggered (such as suspending the conveyor belt or issuing an alarm). Using these implementations, dynamic body movement can be detected in real time based on image and thermal data, thereby improving the safety and security level of the object transport system.
[0051] In this disclosure, the object transport system can be managed based on detected bodies. For example, the transport equipment can be paused once a body intrusion is detected. Alternatively and / or additionally, alarm signals can be provided to notify passengers of the risk situation. Alternatively and / or additionally, the body can be highlighted in image data to notify the object transport system administrator and / or passengers.
[0052] For more details, please refer to Figure 6 , Figure 6 An example interface 600 of a management system for an object transfer system according to an implementation of this disclosure is schematically shown. For example... Figure 6 As shown, interface 610 can be provided to an administrator. Within interface 610, alarms 620 can be added to the video to notify the administrator of dangerous situations. Furthermore, interface 610 can provide various controls to manipulate the video; for example, control 630 can play the video backward, control 632 can pause the video, and control 634 can play the video forward. Using these implementations, the administrator can understand the transmitter's status information at various points in time during operation, thereby improving management efficiency.
[0053] In this disclosed implementation, video can be provided to passengers. See also: [link to relevant documentation] for more details. Figure 7 , Figure 7 An object transport environment 700 according to an implementation of this disclosure is schematically illustrated. For example... Figure 7 As shown, image camera 210 and thermal camera 220 can be equipped near conveyor 740.
[0054] In embodiments of this disclosure, the acquisition area of the image camera overlaps with the acquisition area of the thermal camera, and the image camera includes multiple image cameras and the thermal camera includes multiple thermal cameras. Specifically, the field of view (FoV) of the image camera 210 and the FoV of the thermal camera 220 can overlap, so image data and thermal data can be acquired for the same area, and therefore body detection can be more accurate by means of thermal data. Although this disclosure describes a pair of image cameras 210 and thermal cameras 220 as example cameras, multiple pairs of image cameras and thermal cameras can be deployed in the object transport system 250, and the image cameras and thermal cameras in each pair can have overlapping FoVs.
[0055] In the implementation of this disclosure, each pair of image cameras and thermal cameras is deployed at a predetermined angle. Specifically, one pair of image cameras and thermal cameras can face forward to monitor for physical intrusion on the conveyor, for example, detecting an adult falling onto the conveyor. Simultaneously, another pair of cameras can face downward to collect information about the ground near the conveyor, for example, detecting a child attempting to climb onto the conveyor. Using these implementations, multiple pairs of cameras can be deployed at different angles to detect intruders entering the conveyor system or accidents occurring at any location, further improving detection accuracy.
[0056] Furthermore, a display device 710 can be provided in the object transport system, and the captured video can be presented on the display device 710 along with the identified body. Additionally, a speaker 720 and an alarm device 730 can be provided to notify passengers of potential risks. Using these implementations, the object transport system 250 can be managed automatically with less manual workload.
[0057] In embodiments of this disclosure, the machine learning model may further include a baggage management module. The baggage management module can obtain baggage information about passengers from the airline system (e.g., from the Departure Control System (DCS)). The baggage management module can detect baggage from image data and compare the detected baggage with the detected passengers based on the baggage information. If the baggage belongs to a passenger and is near the passenger on the conveyor belt, the baggage can be highlighted to notify the passenger to claim their baggage nearby.
[0058] In this disclosed implementation, an image camera and a thermal camera can be installed at the entrance of the baggage conveyor belt to ensure that image and thermal data can be acquired using appropriate FoV (Field of View). After installation, both cameras can be calibrated, and the video and thermal signals can then be continuously fed into a machine learning model (e.g., deployed in an edge device) to identify the shape and movement of a human / animal. Specifically, if a potential intrusion is detected, the presence of a human / animal can be confirmed using thermal data through 2D shape and thermal feature analysis. An alarm can then be triggered to notify administrators and / or passengers. For example, the conveyor belt can pause for a period of time to take appropriate action.
[0059] Using the implementation of this disclosure, the presence of humans on a baggage claim area can be detected using video analytics and thermal imaging without interfering with the normal operation of the conveyor. Specifically, body movements can be identified in different postures, and real-time alerts can be provided to administrators and / or passengers in visual, auditory, or mobile device notification formats.
[0060] Although this disclosure describes a baggage handling system in an airport as an example, the object handling system may also include a parcel handling system in a parcel delivery environment. For example, imaging cameras and thermal cameras may be deployed in a parcel delivery environment for body detection.
[0061] The preceding paragraphs have described the details of managing an object delivery system based on body detection. According to an implementation of this disclosure, a method for managing an object delivery system based on body detection is provided. Further details regarding this method will be found in [reference needed]. Figure 8 ,in Figure 8 A flowchart of a method 800 for managing an object transport system based on body detection, according to an implementation of this disclosure, is illustrated schematically. At block 810, image data is received from an image camera and thermal data is received from a thermal camera, both located within a predetermined area around a transport device in the object transport system. At block 820, a machine learning model detects bodies within the predetermined area based on the image data and thermal data. At block 830, the object transport system is managed based on the detected bodies.
[0062] In some implementations of this disclosure, detecting a body within a predetermined area by a machine learning model based on image data and thermal data includes: identifying at least one body region by a machine learning model based on image data and thermal data; and determining a body by a machine learning model based on at least one body region in response to determining that at least one body region belongs to the same body.
[0063] In some implementations of this disclosure, identifying at least one body region from image data and thermal data by a machine learning model includes: identifying at least one candidate image region associated with the body from image data by a machine learning model; identifying at least one candidate thermal region associated with the body from thermal data by a machine learning model; and determining at least one body region based on at least one candidate image region and at least one candidate thermal region by a machine learning model.
[0064] In some implementations of this disclosure, detecting a body within a predetermined area using a machine learning model based on image data and thermal data includes: obtaining the ambient temperature of the object transport system; determining image weights for the image data and thermal weights for the thermal data using a machine learning model; and determining the body using a machine learning model based on the image weights, image data, thermal weights, and thermal data.
[0065] In some implementations of this disclosure, at least one body region is identified as belonging to the same body in response to the following: the distribution of at least one body region satisfies a predetermined body posture; and the total area of at least one body region is greater than a predetermined ratio of the body area.
[0066] In some implementations of this disclosure, the image data is image frames from an image sequence acquired by an image camera, the thermal data is thermal frames from a thermal sequence acquired by a thermal camera, and the body is detected in another image frame in the image sequence and another thermal frame in the thermal sequence.
[0067] In some implementations of this disclosure, managing an object transport system based on a body includes: determining the relationship between a body and a transport device; and managing the object transport system in response to determining that the relationship meets predetermined risk criteria.
[0068] In some implementations of this disclosure, the managed object transport system includes any of the following: pausing the transport equipment; providing an alarm signal; or indicating the body in image data.
[0069] In some implementations of this disclosure, the object delivery system includes any one of the following: a baggage delivery system in a transportation environment, or a parcel delivery system in a parcel delivery environment.
[0070] In some implementations of this disclosure, the acquisition area of the image camera overlaps with the acquisition area of the thermal camera, and the image camera includes multiple image cameras and the thermal camera includes multiple thermal cameras.
[0071] According to an implementation of this disclosure, an apparatus for managing an object transport system based on body detection is provided. The apparatus includes: a receiving module configured to receive image data from an image camera and thermal data from a thermal camera, respectively, within a predetermined area surrounding a transport device in the object transport system; a detection module configured to detect bodies within the predetermined area using a machine learning model based on the image data and thermal data; and a management module configured to manage the object transport system based on the detected bodies. The apparatus also includes other modules configured to implement other steps in the method described above.
[0072] According to an implementation of this disclosure, an electronic device is provided for implementing method 800. The electronic device includes a computer processor coupled to a computer-readable storage unit, the storage unit including instructions that, when executed by the computer processor, implement a method for managing an object transport system based on body detection. Specifically, the processor is configured to cause the device to: receive image data from an image camera and thermal data from a thermal camera, respectively, within a predetermined area surrounding a transport device in the object transport system; detect bodies within the predetermined area by a machine learning model based on the image data and the thermal data; and manage the object transport system based on the detected bodies.
[0073] In some implementations of this disclosure, the electronic device is also configured to: identify at least one body region by a machine learning model based on image data and thermal data; and, in response to determining that at least one body region belongs to the same body, determine the body by the machine learning model based on at least one body region.
[0074] In some implementations of this disclosure, the electronic device is also configured to: identify at least one candidate image region associated with the body from image data by a machine learning model; identify at least one candidate thermal region associated with the body from thermal data by a machine learning model; and determine at least one body region based on at least one candidate image region and at least one candidate thermal region by a machine learning model.
[0075] In some implementations of this disclosure, the electronic device is also configured to: obtain the ambient temperature of the object transport system; determine the image weights of the image data and the thermal weights of the thermal data by a machine learning model; and determine the body by a machine learning model based on the image weights, image data, thermal weights, and thermal data.
[0076] In some implementations of this disclosure, the electronic device is also configured to: determine that at least one body region belongs to the same body in response to: the distribution of at least one body region satisfying a predetermined body posture; and the total area of at least one body region being greater than a predetermined ratio of the body area.
[0077] In some implementations of this disclosure, the image data is image frames from an image sequence acquired by an image camera, the thermal data is hot frames from a thermal sequence acquired by a thermal camera, and the body is detected in another image frame in the image sequence and another hot frame in the thermal sequence.
[0078] In some implementations of this disclosure, the electronic device is also configured to: determine the relationship between the body and the transmission device; and manage the object transmission system in response to determining that the relationship meets a predetermined risk criterion.
[0079] In some implementations of this disclosure, the electronic device is also configured to: suspend the transmission device; provide an alarm signal; or indicate the body in image data.
[0080] In some implementations of this disclosure, the object delivery system includes any one of the following: a baggage delivery system in a transportation environment, or a parcel delivery system in a parcel delivery environment.
[0081] In some implementations of this disclosure, the acquisition area of the image camera overlaps with the acquisition area of the thermal camera, and the image camera includes multiple image cameras and the thermal camera includes multiple thermal cameras.
[0082] According to an implementation of this disclosure, a non-transitory computer program product is provided, comprising a computer-readable storage medium having program instructions embodied thereon, which are executable by an electronic device to cause the electronic device to perform a method for managing an object transport system based on body detection. The method includes: receiving image data from an image camera and thermal data from a thermal camera, respectively, within a predetermined area surrounding a transport device in the object transport system; detecting a body within the predetermined area by a machine learning model based on the image data and thermal data; and managing the object transport system based on the detected body.
[0083] Figure 9 A block diagram of a computing device 900 in which various implementations of the present disclosure may be implemented is shown. It should be understood that... Figure 9 The computing device 900 shown is for illustrative purposes only and does not imply any limitation on the functionality and scope of this disclosure. The computing device 900 can be used to implement the methods described in the embodiments of this disclosure. Figure 9 As shown, the computing device 900 can be a general-purpose computing device. The computing device 900 may include at least one or more processors or processing units 910, memory 920, storage unit 930, one or more communication units 940, one or more input devices 950, and one or more output devices 960.
[0084] Processing unit 910 can be a physical or virtual processor and can implement various processes based on programs stored in memory 920. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capabilities of computing device 900. Processing unit 910 may also be referred to as a central processing unit (CPU), microprocessor, controller, or microcontroller.
[0085] Computing device 900 typically includes various computer storage media. Such media can be any media accessible by computing device 900, including but not limited to volatile and non-volatile media, or removable and non-removable media. Memory 720 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), or flash memory) or any combination thereof. Storage cell 930 can be any removable or non-removable media and may include machine-readable media, such as memory, flash drives, disks, or other media, which can be used to store information and / or data and can be accessed within computing device 900.
[0086] The computing device 900 may also include additional removable / non-removable, volatile / non-volatile memory media. Although in Figure 9 Not shown, but may include disk drives for reading from and / or writing to removable non-volatile disks and optical disc drives for reading from and / or writing to removable non-volatile optical discs. In this case, each drive may be connected to a bus (not shown) via one or more data media interfaces.
[0087] The communication unit 940 communicates with another computing device via a communication medium. Furthermore, the functionality of the components in the computing device 900 can be implemented by a single computing cluster or multiple computing machines that can communicate via communication connections. Therefore, the computing device 900 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general-purpose network nodes.
[0088] Input device 950 can be one or more of various input devices, such as a mouse, keyboard, trackball, voice input device, etc. Output device 960 can be one or more of various output devices, such as a monitor, speaker, printer, etc. With the aid of communication unit 940, computing device 900 can also communicate with one or more external devices (not shown) (such as storage devices and display devices), one or more of which enable a user to interact with computing device 900, or, if needed, any device (such as a network card, modem, etc.) enables computing device 900 to communicate with one or more other computing devices. This communication can be performed via input / output (I / O) interface (not shown).
[0089] In some implementations, some or all components of computing device 900 may be deployed within a cloud computing architecture, rather than integrated into a single device. In a cloud computing architecture, components can be remotely provided and work together to achieve the functionality described in this disclosure. In some implementations, cloud computing provides computing, software, data access, and storage services without requiring end users to know the physical location or configuration of the systems or hardware providing these services. In various implementations, cloud computing provides services via a wide area network (WAN), such as the Internet, using appropriate protocols. For example, a cloud computing provider offers applications via a WAN that can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture, along with the corresponding data, may be stored on servers at remote locations. Computing resources in a cloud computing environment may be consolidated or distributed across multiple locations in remote data centers. Cloud computing infrastructure can provide services through shared data centers, although they act as a single access point for users. Therefore, cloud computing architectures can be used to provide the components and functionality described herein from service providers at remote locations. Alternatively, they may be provided from conventional servers or directly installed or otherwise installed on client devices.
[0090] The functions described herein can be performed at least in part by one or more hardware logic components. For example, but not limited to, illustrative types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), etc.
[0091] Program code used to perform the methods described herein can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code enables the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely or partially on the machine, partially as a standalone software package on the machine, partially on a remote machine, or entirely on a remote machine or server.
[0092] In the context of this disclosure, a machine-readable medium can be any tangible medium that can contain or store a program used by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More specific examples of machine-readable storage media will include electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0093] Furthermore, although operations are shown in a specific order, this should not be construed as requiring that such operations be performed in the specific order shown or sequentially, or that all shown operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the topics described herein, but rather as descriptions of features that may be specific to a particular implementation. Certain features described in the context of a single implementation may also be implemented in combination within a single implementation. Conversely, various features described in a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.
[0094] Although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter specified in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as exemplary forms for implementing the claims.
[0095] Based on the foregoing, it should be understood that this document has described specific implementations of the currently disclosed technology for illustrative purposes, but various modifications can be made without departing from the scope of this disclosure. Therefore, the technology disclosed herein is not limited except for the appended claims.
[0096] The subject matter and functional operations described in this disclosure can be implemented in various systems, digital electronic circuits, or computer software, firmware, or hardware (including the structures disclosed in this specification and their equivalents), or combinations thereof. Implementations of the subject matter described in this specification can be carried out as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory computer-readable medium for execution by or control of the operation of a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of materials influencing machine-readable propagation signals, or combinations thereof. The terms "data processing unit" or "data processing apparatus" encompass all means, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or combinations thereof.
[0097] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suited to a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), as a single file dedicated to the program in question, or as multiple coordinating files (e.g., files storing one or more modules, subroutines, or code sections). Computer programs can be deployed to execute on one or more computers located at a single site or distributed across multiple sites and interconnected via a communication network.
[0098] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more processors in any type of digital computer. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to, receiving data from or transferring data to, or both to, one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data. However, a computer does not need to have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices. The processor and memory may be supplemented by or incorporated into special-purpose logic circuitry.
[0099] The specification together with the accompanying drawings is intended to be considered exemplary only, where exemplary means example. As used herein, the use of "or" is intended to include "and / or" unless the context clearly indicates otherwise.
[0100] While this disclosure contains numerous details, these details should not be construed as limiting the scope of any disclosure or claimable content, but rather as descriptions of features specific to particular implementations of a particular disclosure. Certain features described in this disclosure in the context of a single implementation may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations. Furthermore, although features may be described above as functioning in certain combinations and even initially claimed in this way, in some cases, one or more features from a claimed combination may be removed from the combination, and the claimed combination may involve sub-combinations or variations thereof.
[0101] Similarly, although operations are shown in a specific order in the accompanying drawings, this should not be construed as requiring the operations to be performed in the specific order shown or sequentially, or to perform all shown operations to achieve the desired result. Furthermore, the separation of various system components in the implementations described in this disclosure should not be construed as requiring such separation in all implementations. Only some implementations and examples have been described, and other implementations, enhancements, and variations can be made based on what is described and shown in this disclosure.
Claims
1. A method for managing an object delivery system based on body detection, comprising: Image data from an image camera and thermal data from a thermal camera are received respectively, the image camera and the thermal camera being located within a predetermined area around the conveyor equipment in the object conveying system; The body within the predetermined area is detected by a machine learning model based on the image data and the thermal data; as well as The object delivery system is managed based on the detected body.
2. The method of claim 1, wherein detecting the body within the predetermined region by the machine learning model based on the image data and the thermal data comprises: The machine learning model identifies at least one body region based on the image data and the thermal data; as well as In response to determining that the at least one body region belongs to the same body, the machine learning model determines the body based on the at least one body region.
3. The method of claim 2, wherein identifying the at least one body region from the image data and the thermal data by the machine learning model comprises: The machine learning model identifies at least one candidate image region associated with the body from the image data; The machine learning model identifies at least one candidate thermal region associated with the body from the thermal data; as well as The machine learning model determines the at least one body region based on the at least one candidate image region and the at least one candidate hot region.
4. The method of claim 1, wherein detecting the body within the predetermined region by the machine learning model based on the image data and the thermal data comprises: Obtain the ambient temperature of the object transport system; The machine learning model determines the image weights for the image data and the heat weights for the heat data, respectively. as well as The machine learning model determines the body based on the image weights, the image data, the thermal weights, and the thermal data.
5. The method of claim 3, wherein the at least one body region is determined to belong to the same body in response to: The distribution of the at least one body region satisfies a predetermined body posture; and The total area of at least one body region is higher than a predetermined ratio of the body area.
6. The method of claim 1, wherein the image data is image frames from an image sequence acquired by the image camera, the thermal data is thermal frames from a thermal sequence acquired by the thermal camera, and the body is detected in another image frame in the image sequence and another thermal frame in the thermal sequence.
7. The method of claim 1, wherein managing the object delivery system based on the body comprises: Determine the relationship between the body and the conveyor equipment; as well as In response to determining that the relationship meets a predetermined risk criterion, the object transfer system is managed.
8. The method of claim 1, wherein managing the object transfer system comprises any one of the following: Pause the conveyor equipment; Provide alarm signals; or The body is indicated in the image data.
9. The method of claim 1, wherein the object delivery system comprises any one of the following: a baggage delivery system in a transportation environment, or a parcel delivery system in a parcel delivery environment.
10. The method of claim 1, wherein the acquisition area of the image camera overlaps with the acquisition area of the thermal camera, the image camera comprising a plurality of image cameras and the thermal camera comprising a plurality of thermal cameras.
11. An electronic device, comprising: Processor, the processor being configured to cause the device to: Image data from an image camera and thermal data from a thermal camera are received respectively, and the image camera and the thermal camera are located within a predetermined area around the conveyor equipment in the object conveying system; The body within the predetermined area is detected by a machine learning model based on the image data and the thermal data; as well as The object delivery system is managed based on the detected body.
12. The device of claim 11, wherein the electronic device is further configured to: The machine learning model identifies at least one body region based on the image data and the thermal data; and In response to determining that the at least one body region belongs to the same body, the machine learning model determines the body based on the at least one body region.
13. The device of claim 12, wherein the electronic device is further configured to: The machine learning model identifies at least one candidate image region associated with the body from the image data; The machine learning model identifies at least one candidate thermal region associated with the body from the thermal data; and The machine learning model determines the at least one body region based on the at least one candidate image region and the at least one candidate hot region.
14. The device of claim 11, wherein the electronic device is further configured to: Obtain the ambient temperature of the object transport system; The machine learning model determines the image weights for the image data and the heat weights for the heat data, respectively. as well as The machine learning model determines the body based on the image weights, the image data, the thermal weights, and the thermal data.
15. The device of claim 13, wherein the electronic device is further configured to: The at least one body region is determined to belong to the same body in response to the following: The distribution of the at least one body region satisfies a predetermined body posture; and The total area of at least one body region is higher than a predetermined ratio of the body area.
16. The device of claim 11, wherein the image data is image frames from an image sequence acquired by the image camera, the thermal data is thermal frames from a thermal sequence acquired by the thermal camera, and the body is detected in another image frame of the image sequence and another thermal frame of the thermal sequence.
17. The device of claim 11, wherein the electronic device is further configured to: Determine the relationship between the body and the conveyor equipment; and In response to determining that the relationship meets a predetermined risk criterion, the object transfer system is managed.
18. The device of claim 11, wherein the electronic device is further configured to: Pause the conveyor equipment; Provide alarm signals; or The body is indicated in the image data.
19. The apparatus of claim 11, wherein the object conveying system comprises any one of: a baggage conveying system in a transportation environment, or a parcel conveying system in a parcel delivery environment, wherein the acquisition area of the image camera overlaps with the acquisition area of the thermal camera, the image camera comprising a plurality of image cameras and the thermal camera comprising a plurality of thermal cameras.
20. A non-transitory computer program product comprising a computer-readable storage medium having program instructions embodied thereon, the program instructions being executable by an electronic device to cause the electronic device to perform a method for managing an object delivery system based on body detection, the method comprising: Image data from an image camera and thermal data from a thermal camera are received respectively, the image camera and the thermal camera being located within a predetermined area around the conveyor equipment in the object conveying system; The body within the predetermined area is detected by a machine learning model based on the image data and the thermal data; as well as The object delivery system is managed based on the detected body data.
