Conveyor belt control method and device, electronic equipment and computer readable medium

Abstract

The invention discloses a conveyor belt control method and device, electronic equipment and a computer readable medium, and relates to the technical field of computers.The specific implementation mode comprises the steps that monitoring data obtained by monitoring a target area of a target conveyor belt is obtained; congestion sensing is carried out on the monitoring data to determine the current cargo conveying state of the target area; inputting the cargo conveying state to a conveyor belt control decision model to obtain a long-term benefit brought by executing a preset conveyor belt control action in the cargo conveying state; and determining a conveyor belt control target action according to the long-term income, and executing the conveyor belt control target action. Therefore, after the congestion condition is recognized, the operation state of the conveyor belt is adjusted by executing the determined conveyor belt control target action, the danger easily generated when the conveyor belt congestion is manually handled can be effectively avoided, the problem of conveyor belt congestion can be fundamentally solved, and the efficiency and accuracy of conveyor belt control are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a conveyor belt control method, apparatus, electronic device, and computer-readable medium. Background Technology

[0002] Sorting is a crucial part of the logistics process, especially in large sorting centers. Large volumes of goods need to be diverted using automated equipment and a certain amount of manpower, ultimately arriving at pre-designated locations to be loaded onto trucks and transported to corresponding distribution stations and other sorting centers. However, sorting centers typically handle large volumes of goods, while the capacity of sorting equipment is limited. Furthermore, due to time constraints, multiple vehicles often arrive simultaneously for unloading, causing congestion at the merging points.

[0003] Currently, most methods, after identifying congestion, directly address the location of the congestion. Firstly, the complex structure of conveyor belts makes many locations difficult for workers to reach, or potentially dangerous. Secondly, while manual intervention can temporarily relieve localized congestion, it doesn't address the root cause, and the congestion will persist. Furthermore, conveyor belt control is inefficient and inaccurate. Summary of the Invention

[0004] In view of this, embodiments of this application provide a conveyor belt control method, device, electronic device, and computer-readable medium, which can solve the problems of existing methods that directly process the location of congestion after identifying congestion. Due to the complex structure of the conveyor belt, many locations are difficult for workers to reach or are prone to danger. Although manual processing can temporarily relieve local congestion, it does not solve the problem at its root and the congestion will continue. Furthermore, the conveyor belt control is inefficient and inaccurate.

[0005] To achieve the above objectives, according to one aspect of the embodiments of this application, a conveyor belt control method is provided, comprising:

[0006] Acquire monitoring data obtained from monitoring the target area of ​​the target conveyor belt;

[0007] Use monitoring data to detect congestion and determine the current cargo delivery status in the target area;

[0008] The cargo conveying status is input into the conveyor belt control decision model to obtain the long-term benefits of executing preset conveyor belt control actions under the cargo conveying status.

[0009] Determine the target action for conveyor belt control based on long-term benefits, and execute the target action for conveyor belt control.

[0010] Optionally, congestion sensing is performed on the monitoring data to determine the cargo delivery status corresponding to the monitoring data, including:

[0011] Feature extraction is performed on each frame of the monitoring data to obtain the feature sequence corresponding to each frame;

[0012] The feature sequence corresponding to each frame is input into the encoding model to obtain an encrypted feature sequence that combines the information of the previous and next frames.

[0013] The encrypted feature sequence corresponding to each frame is concatenated according to the time sequence to obtain the concatenated encrypted feature sequence;

[0014] The concatenated encrypted feature sequence is used to determine the current cargo delivery status in the target area.

[0015] Optionally, the preset conveyor belt control actions include maintaining the status quo, starting a conveyor belt, and stopping a conveyor belt;

[0016] The cargo conveying status is input into the conveyor belt control decision model to obtain the long-term benefits of executing preset conveyor belt control actions under the cargo conveying status, including:

[0017] The cargo transport status is input into the conveyor belt control decision model to obtain the long-term benefits of maintaining the status quo, starting a conveyor belt, and shutting down a conveyor belt, as output by the conveyor belt control decision model.

[0018] Optionally, the target action for conveyor belt control is determined based on long-term benefits, and the target action for conveyor belt control is executed, including:

[0019] The preset conveyor belt control action corresponding to the greatest long-term benefit is determined as the conveyor belt control target action;

[0020] Identify the conveyor belt that enters the target area corresponding to the conveyor belt control target action, and execute the conveyor belt control target action on the conveyor belt.

[0021] Optionally, the conveyor belt control method further includes:

[0022] The sample cargo transport status is set in the sample target area of ​​the sample target conveyor belt that has collected samples;

[0023] The system is configured to collect and execute sample conveyor belt control actions during sample cargo transport: maintain the status quo, start one conveyor belt, and shut down one conveyor belt;

[0024] A long-term benefit for the sample is set up, which is collected based on the following method: historical operation records of the target conveyor belts are analyzed. For each conveyor belt closed, a preset first negative benefit is added; for each conveyor belt opened, a corresponding positive benefit is added; if the status quo is maintained or a conveyor belt is opened, then if congestion occurs within a preset number of time units after the conveyor belt control action, a preset second negative benefit is added. The sum of the positive and negative benefits obtained is taken as the long-term benefit of the sample.

[0025] Based on the above settings, the conveyor belt control decision model is trained by considering the sample cargo transport state, the sample conveyor belt control actions, and the long-term benefits of executing the sample conveyor belt control actions under a given sample cargo transport state.

[0026] Optionally, the conveyor belt control method further includes:

[0027] Determine the corresponding priority for all conveyor belts that receive samples into the target area based on timeliness requirements;

[0028] Activate a conveyor belt, including: activating the conveyor belt with the highest priority;

[0029] Shutting down a conveyor belt includes shutting down the lowest priority conveyor belt.

[0030] In addition, this application also provides a conveyor belt control device, including:

[0031] The monitoring data acquisition unit is configured to acquire monitoring data obtained by monitoring the target area of ​​the target conveyor belt;

[0032] The congestion sensing unit is configured to sense congestion in monitoring data in order to determine the current cargo delivery status in the target area.

[0033] The long-term benefit determination unit is configured to input the cargo conveying state into the conveyor belt control decision model in order to obtain the long-term benefits brought about by performing preset conveyor belt control actions in the cargo conveying state.

[0034] The conveyor belt control unit is configured to determine the target conveyor belt control action based on long-term benefits and execute the target conveyor belt control action.

[0035] Optionally, the congestion sensing unit is further configured to:

[0036] Feature extraction is performed on each frame of the monitoring data to obtain the feature sequence corresponding to each frame;

[0037] The feature sequence corresponding to each frame is input into the encoding model to obtain an encrypted feature sequence that combines the information of the previous and next frames.

[0038] The encrypted feature sequence corresponding to each frame is concatenated according to the time sequence to obtain the concatenated encrypted feature sequence;

[0039] The concatenated encrypted feature sequence is used to determine the current cargo delivery status in the target area.

[0040] Optionally, the preset conveyor belt control actions include maintaining the status quo, starting a conveyor belt, and stopping a conveyor belt;

[0041] The long-term return determination unit is further configured as follows:

[0042] The cargo transport status is input into the conveyor belt control decision model to obtain the long-term benefits of maintaining the status quo, starting a conveyor belt, and shutting down a conveyor belt, as output by the conveyor belt control decision model.

[0043] Optionally, the conveyor belt control unit is further configured to:

[0044] The preset conveyor belt control action corresponding to the greatest long-term benefit is determined as the conveyor belt control target action;

[0045] Identify the conveyor belt that enters the target area corresponding to the conveyor belt control target action, and execute the conveyor belt control target action on the conveyor belt.

[0046] Optionally, the conveyor belt control device also includes a model training unit, configured to:

[0047] The sample cargo transport status is set in the sample target area of ​​the sample target conveyor belt that has collected samples;

[0048] The system is configured to collect and execute sample conveyor belt control actions during sample cargo transport: maintain the status quo, start one conveyor belt, and shut down one conveyor belt;

[0049] A long-term benefit for the sample is set up, which is collected based on the following method: historical operation records of the target conveyor belts are analyzed. For each conveyor belt closed, a preset first negative benefit is added; for each conveyor belt opened, a corresponding positive benefit is added; if the status quo is maintained or a conveyor belt is opened, then if congestion occurs within a preset number of time units after the conveyor belt control action, a preset second negative benefit is added. The sum of the positive and negative benefits obtained is taken as the long-term benefit of the sample.

[0050] Based on the above settings, the conveyor belt control decision model is trained by considering the sample cargo transport state, the sample conveyor belt control actions, and the long-term benefits of executing the sample conveyor belt control actions under a given sample cargo transport state.

[0051] Optionally, the model training unit is further configured to:

[0052] Determine the corresponding priority for all conveyor belts that receive samples into the target area based on timeliness requirements;

[0053] The model training unit is further configured to activate the highest priority conveyor belt;

[0054] The model training unit is further configured to shut down the conveyor belt with the lowest priority.

[0055] In addition, this application also provides a conveyor belt control electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the conveyor belt control method as described above.

[0056] In addition, this application also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the conveyor belt control method as described above.

[0057] To achieve the above objectives, according to another aspect of the embodiments of this application, a computer program product is provided.

[0058] A computer program product according to an embodiment of this application includes a computer program that, when executed by a processor, implements the conveyor belt control method provided in an embodiment of this application.

[0059] One embodiment of the above invention has the following advantages or beneficial effects: This application obtains monitoring data by monitoring the target area of ​​the target conveyor belt; performs congestion perception on the monitoring data to determine the current cargo conveying status of the target area; inputs the cargo conveying status into the conveyor belt control decision model to obtain the long-term benefits of executing preset conveyor belt control actions under the cargo conveying status; determines the conveyor belt control target action based on the long-term benefits, and executes the conveyor belt control target action. Therefore, after identifying congestion, adjusting the operating status of the conveyor belt by executing the determined conveyor belt control target action can effectively avoid the dangers that are easily caused by manually handling conveyor belt congestion, and can solve the conveyor belt congestion problem at its root, improving the efficiency and accuracy of conveyor belt control.

[0060] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0061] The accompanying drawings are provided to better understand this application and do not constitute an undue limitation thereof. Wherein:

[0062] Figure 1This is a schematic diagram of the main flow of a conveyor belt control method according to an embodiment of this application;

[0063] Figure 2 This is a schematic diagram of the main flow of a conveyor belt control method according to an embodiment of this application;

[0064] Figure 3 This is a schematic diagram of congestion identification and PLC conveyor belt operation control according to an embodiment of the conveyor belt control method of this application;

[0065] Figure 4 This is a flow chart of the congestion sensing module of a conveyor belt control method according to an embodiment of this application;

[0066] Figure 5 This is a schematic diagram of a conveyor belt control decision model according to an embodiment of the conveyor belt control method of this application;

[0067] Figure 6 This is a schematic diagram of the main units of a conveyor belt control device according to an embodiment of this application;

[0068] Figure 7 This is an exemplary system architecture diagram to which embodiments of this application can be applied;

[0069] Figure 8 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers in the embodiments of this application. Detailed Implementation

[0070] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These embodiments should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solutions of this application comply with relevant national laws and regulations. It should also be noted that certain software, components, models, and other existing industry solutions may be mentioned in the embodiments of this application. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solutions of this application, and do not imply that the applicant has already used or necessarily used such solutions. The collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in the technical solutions of this application all comply with relevant laws and regulations, are used for legal and reasonable purposes, do not violate public order and good morals, are not shared, disclosed, or sold outside of these legal uses, and are subject to supervision and management by regulatory authorities. Necessary measures should be taken to prevent unauthorized access to user personal information, safeguard user personal information security, cybersecurity, and national security, and ensure that those authorized to access personal information comply with relevant laws and regulations. Once this user personal information is no longer needed, risks should be minimized by restricting or even prohibiting data collection and / or deleting the data.

[0071] When used, including in certain relevant applications, data is deidentified to protect user privacy, for example by removing specific identifiers, controlling the amount or specificity of stored data, controlling how data is stored, and / or other methods.

[0072] Figure 1 This is a schematic diagram of the main flow of a conveyor belt control method according to an embodiment of this application, as shown below. Figure 1 As shown, the conveyor belt control method mainly includes the following steps S101-S104.

[0073] Step S101: Obtain monitoring data obtained by monitoring the target area of ​​the target conveyor belt.

[0074] In this embodiment, the execution entity of the conveyor belt control method (e.g., a server in a PLC control device) can communicate with the monitoring camera via a wired or wireless connection to obtain monitoring data obtained by the monitoring camera monitoring the target area (i.e., the area to be monitored) of the target conveyor belt (i.e., the conveyor belt to be monitored). This monitoring data can be images and / or videos captured within a preset time period.

[0075] Step S102: Perform congestion detection on the monitoring data to determine the current cargo delivery status in the target area.

[0076] By performing congestion perception on monitoring data, the spatiotemporal features of each frame in the monitoring data can be extracted, spatiotemporal features can be stitched together, and then transformed into a spatiotemporal feature vector. This spatiotemporal feature vector can be used as the current cargo conveying status of the target area of ​​the target conveyor belt, which can be directly used by the conveyor belt control decision model to make decisions on conveyor belt control.

[0077] Step S103: Input the cargo conveying status into the conveyor belt control decision model to obtain the long-term benefits brought about by executing preset conveyor belt control actions in the cargo conveying status.

[0078] Specifically, the preset conveyor belt control actions include maintaining the status quo, starting one conveyor belt, and stopping one conveyor belt; the goods conveying status is input into the conveyor belt control decision model to obtain the long-term benefits brought about by performing the preset conveyor belt control actions in the goods conveying status, including: inputting the goods conveying status into the conveyor belt control decision model to obtain the long-term benefits brought about by maintaining the status quo, starting one conveyor belt, and stopping one conveyor belt output by the conveyor belt control decision model in the goods conveying status.

[0079] Specifically, the conveyor belt control method also includes: setting the sample cargo conveying state of the sample target area of ​​the sample target conveyor belt; for example, State (i.e., sample cargo conveying state): the State output by the congestion sensing module can be directly used to represent the overall situation of the current monitoring area; setting the sample conveyor belt control actions collected and executed in the sample cargo conveying state: maintain the status quo, start one conveyor belt, and close one conveyor belt; for example, Action (i.e., sample conveyor belt control action): such as Figure 3 As shown, it is necessary to monitor the conveyor belts (e.g., those entering the monitoring area, i.e., the target area of ​​this application) that are connected to the monitoring area. Figure 3The system adjusts one or more of the conveyor belts corresponding to the main line, branch lines, and U-branch lines. Each conveyor belt has two actions: On (open) and Off (close). Since multiple conveyor belts may converge, the actual action is a combination of all conveyor belt actions. To simplify the action space, this solution prioritizes all converging conveyor belts based on their timeliness requirements; that is, conveyor belts with higher timeliness requirements have higher priority. When it is necessary to close a conveyor belt to alleviate congestion, lower-priority conveyor belts are closed first. Conversely, when it is necessary to open a conveyor belt to improve timeliness, higher-priority conveyor belts are opened first. Therefore, this solution designs three types of actions, using... This indicates that the options represent: maintaining the status quo, turning on one conveyor belt, and turning off one conveyor belt. This approach also facilitates compatibility with different conveyor belt structures. A sample long-term benefit is set, which is collected based on the following method: analyzing historical sample target conveyor belt operation records, each time a conveyor belt is turned off, a preset first negative benefit is added; each time a conveyor belt is turned on, a corresponding positive benefit is added. If the option of maintaining the status quo or turning on one conveyor belt is chosen, then if congestion occurs within a preset number of time units after the conveyor belt control action, a preset second negative benefit is added. The sum of the final positive and negative benefits is taken as the sample long-term benefit. For example, the Reward (i.e., the sample long-term benefit): The Reward needs to include both the losses caused by congestion and the benefits brought by improved timeliness. Each time a conveyor belt is turned off, a negative benefit of -R (i.e., the first negative benefit) is generated; conversely, each time a conveyor belt is turned on, a corresponding positive benefit of R is generated. If the current situation is maintained or a conveyor belt is activated, then if congestion occurs within M time units (i.e., a preset number of time units) after activation, the corresponding action will receive a negative reward of -E (i.e., a second negative reward). The total reward (i.e., the long-term sample reward) is a combination of the above factors. Based on the above settings, a conveyor belt control decision model is trained by considering the sample cargo transport state, sample conveyor belt control actions, and the long-term sample reward resulting from executing the sample conveyor belt control actions under a given sample cargo transport state. For example, the model is trained using the sample cargo transport state and sample conveyor belt control actions as input, and the long-term sample reward resulting from executing the sample conveyor belt control actions under a given sample cargo transport state as the model's expected output.

[0080] Specifically, the conveyor belt control method also includes: determining the corresponding priority of all conveyor belts entering the target area of ​​the sample according to the timeliness requirements; starting a conveyor belt, including: starting the conveyor belt with the highest priority; and stopping a conveyor belt, including: stopping the conveyor belt with the lowest priority.

[0081] For example, during model training, if a conveyor belt needs to be turned on or off, the priority of all conveyor belts flowing into the target area of ​​the sample needs to be determined first, based on the timeliness requirements. Then, based on the determined priorities, it is decided which conveyor belt to turn on or off. For example, if a conveyor belt needs to be turned on, the one with the highest priority will be turned on; if a conveyor belt needs to be turned off, the one with the lowest priority will be turned off, in order to maximize the efficiency of goods transportation, alleviate congestion, and ensure timeliness requirements.

[0082] Step S104: Determine the target action for conveyor belt control based on long-term benefits, and execute the target action for conveyor belt control.

[0083] Specifically, the conveyor belt control target action is determined based on long-term benefits, and then executed. This includes: determining the preset conveyor belt control action corresponding to the greatest long-term benefit as the conveyor belt control target action (e.g., activating a conveyor belt); determining the conveyor belt (e.g., the highest priority conveyor belt) that merges into the target area corresponding to the conveyor belt control target action (e.g., activating a conveyor belt); and executing the conveyor belt control target action (e.g., activating the highest priority conveyor belt) on the conveyor belt (e.g., the highest priority conveyor belt). This alleviates congestion and ensures timely delivery.

[0084] This embodiment acquires monitoring data from the target area of ​​the target conveyor belt; performs congestion detection on the monitoring data to determine the current cargo conveying status of the target area; inputs the cargo conveying status into the conveyor belt control decision model to obtain the long-term benefits of executing preset conveyor belt control actions under the cargo conveying status; determines the target conveyor belt control action based on the long-term benefits, and executes the target conveyor belt control action. Therefore, after identifying congestion, adjusting the conveyor belt's operating status by executing the determined target conveyor belt control action can effectively avoid the dangers that are easily caused by manually handling conveyor belt congestion, and can solve the conveyor belt congestion problem at its root, improving the efficiency and accuracy of conveyor belt control.

[0085] Figure 2 This is a schematic flowchart of a conveyor belt control method according to an embodiment of this application, as shown below. Figure 2 As shown, the conveyor belt control method mainly includes the following steps S201-S207.

[0086] Step S201: Obtain monitoring data obtained by monitoring the target area of ​​the target conveyor belt.

[0087] The target conveyor belt is the conveyor belt that needs to be monitored; the target area is the area that needs to be monitored. The monitoring data obtained from monitoring the target area of ​​the target conveyor belt can include images and / or video.

[0088] Step S202: Extract features from each frame of the monitoring data to obtain the feature sequence corresponding to each frame.

[0089] The monitoring data is segmented into frames to obtain each corresponding frame. Features of each frame are extracted to obtain a corresponding feature sequence. This ensures full utilization of the monitoring data and improves the accuracy of conveyor belt control.

[0090] Step S203: Input the feature sequence corresponding to each frame into the encoding model to obtain an encrypted feature sequence that combines the information of the previous and next frames.

[0091] Encoding models, such as the Transformer Encoder model, convert the feature sequence corresponding to each input frame into a high-dimensional vector representation, thus obtaining an encrypted feature sequence that combines information from previous and subsequent frames.

[0092] Step S204: Concatenate the encrypted feature sequence corresponding to each frame according to the time sequence to obtain the concatenated encrypted feature sequence.

[0093] The corresponding encrypted feature sequences of each frame are spliced ​​together in chronological order from farthest to nearest to obtain a spliced ​​encrypted feature sequence (essentially a vector), which is then used directly in subsequent models to improve the efficiency of conveyor belt control.

[0094] Step S205: Determine the concatenated encrypted feature sequence as the current cargo delivery status of the target area.

[0095] The concatenated encrypted feature sequence is represented in vector form. This concatenated encrypted feature sequence is determined as the current cargo conveying state of the target area, so that the cargo conveying state represented by this vector form can be directly input into the conveyor belt control decision model to make conveyor belt control decisions and improve the efficiency of conveyor belt control.

[0096] Step S206: Input the cargo conveying status into the conveyor belt control decision model to obtain the long-term benefits brought about by executing preset conveyor belt control actions in the cargo conveying status.

[0097] The cargo transport status is input into the conveyor belt control decision model to obtain the long-term benefits of maintaining the status quo, starting one conveyor belt, and shutting down one conveyor belt, as output by the model. (Example provided). This long-term benefit can be a score.

[0098] Step S207: Determine the target action for conveyor belt control based on long-term benefits, and execute the target action for conveyor belt control.

[0099] The preset conveyor belt control action corresponding to the greatest long-term benefit is determined as the conveyor belt control target action (e.g., shutting down a conveyor belt); the conveyor belt entering the target area corresponding to the conveyor belt control target action (e.g., shutting down a conveyor belt) is determined, and the conveyor belt control target action (e.g., shutting down the lowest priority conveyor belt) is executed on the conveyor belt (e.g., the lowest priority conveyor belt). This alleviates congestion and ensures timely delivery.

[0100] PLC linkage primarily achieves coordinated control between devices through communication protocols, signal input / output, and program control. In this application, the problem of congestion on the conveyor belt is alleviated by adjusting its operating state (e.g., opening or closing). After congestion occurs at a location, the operating state of the relevant conveyor belts can be controlled via PLC linkage to reduce the flow rate of goods entering that location, thereby alleviating congestion. The corresponding scheme considers the overall impact of control commands on the conveyor belts, including the mitigation of congestion and the timeliness impact on tasks associated with each relevant conveyor belt, thus providing reasonable control commands. This application uses reinforcement learning to model conveyor belt congestion identification and PLC linkage control, and learns reasonable control strategies from data.

[0101] Figure 3 This is a schematic diagram illustrating congestion identification and PLC conveyor belt operation control according to an embodiment of the conveyor belt control method of this application. Figure 3 As shown, monitoring data of the target area (e.g., area A) of the target conveyor belt is acquired through a set monitoring camera. The monitoring camera is communicatively connected to the PLC control device and transmits the monitoring data to the PLC control device. When congestion is detected in the target area (e.g., area A) based on the monitoring data, the PLC control device can execute the target action of conveyor belt control based on the decision result of the conveyor belt control decision model, that is, send... Figure 3 One or more of the main line, branch line, and U-branch line in the system issue conveyor control commands to adjust the flow rate of goods in one or more of the C area, D area, and B area.

[0102] Figure 4 This application describes the structure and flow of a congestion sensing module in a conveyor belt control method according to one embodiment. This solution uses reinforcement learning to link with a PLC to control the conveyor belt's operating speed, alleviating congestion at key nodes while ensuring the overall operational efficiency of the sorting system. This solution involves two modules: a congestion sensing module and a PLC control module.

[0103] The congestion perception module is implemented using computer vision to detect congestion in monitoring data and determine the current cargo conveying status in the target area. In this solution, the conveyor belt congestion perception method needs to be linked with the PLC control method. This solution uses a neural network structure to extract features from a video segment (i.e., monitoring data) to assist the PLC control module in its judgment. Therefore, this solution uses a monitoring camera to continuously acquire images (i.e., monitoring data) of the area to be monitored (i.e., the target area of ​​the target conveyor belt), and maps several consecutive frames of images to a virtual representation to represent the current situation of that area. For example, this solution designs a corresponding network structure, such as... Figure 4 As shown. For each input frame (each frame refers to a single static image, the smallest unit in a video or animation; playing these static frames consecutively creates a dynamic visual effect), ResNet is used for feature extraction. The feature sequence corresponding to each frame in the video clip is then input into the Transformer Encoder model (the Transformer Encoder is the encoding part of the Transformer model, mainly responsible for converting the input sequence into a high-dimensional vector representation, capturing semantic and structural information in the sequence, and providing contextual information for the decoder). This yields an encrypted feature sequence combining information from preceding and following frames. Finally, these sequences are concatenated, resulting in a vector called State, which represents the current state of the monitored area (i.e., the current cargo transport status of the target area). The model parameters of ResNet and Transformer Encoder are trained by determining whether there is congestion. First, the input video clip is labeled (either manually or using existing algorithms) to obtain two categories: congested and not congested, and then classification training is performed. During training, State is input into a classifier to determine whether there is congestion, but after training, the reinforcement learning input does not require the final classifier.

[0104] The conveyor belt control module is built using reinforcement learning. Based on the State information input from the congestion perception module, it determines how to adjust the flow rate to alleviate congestion on the conveyor belt and ensure efficient transport. The conveyor belt control method can send two commands: conveyor belt off and on. The specific command issuance is handled by the PLC module. The reinforcement learning model (i.e., the conveyor belt control decision model) of the conveyor belt control module is constructed as follows:

[0105] State (i.e., sample cargo delivery status): The State output by the congestion perception module can be used directly to represent the overall situation of the current monitoring area.

[0106] Action (i.e., sample conveyor belt control action): such as Figure 3 As shown, it is necessary to monitor the conveyor belts (e.g., those entering the monitoring area, i.e., the target area of ​​this application) that are connected to the monitoring area. Figure 3 The system adjusts one or more of the conveyor belts corresponding to the main line, branch lines, and U-branch lines. Each conveyor belt has two actions: On (open) and Off (close). Since multiple conveyor belts may converge, the actual action is a combination of all conveyor belt actions. To simplify the action space, this solution prioritizes all converging conveyor belts based on their timeliness requirements; that is, conveyor belts with higher timeliness requirements have higher priority. When it is necessary to close a conveyor belt to alleviate congestion, lower-priority conveyor belts are closed first. Conversely, when it is necessary to open a conveyor belt to improve timeliness, higher-priority conveyor belts are opened first. Therefore, this solution designs three types of actions, using... This indicates that the options represent: maintaining the status quo, turning one conveyor belt on, and turning one conveyor belt off. This method also facilitates compatibility with different conveyor belt structures.

[0107] Reward (i.e., long-term sample benefit): The reward needs to include both the losses caused by congestion and the benefits brought by improved timeliness. Closing a conveyor belt results in a negative benefit -R (the first negative benefit), while opening a conveyor belt results in a corresponding positive benefit R. If the approach is to maintain the status quo or open a conveyor belt, and congestion occurs within M time units (a preset number of time units) after opening, the corresponding action will receive a negative benefit -E (the second negative benefit). The total reward (i.e., long-term sample benefit) is a combination of the above factors.

[0108] Based on the above settings, this solution uses Deep Q-Learning (DQN) to build a reinforcement learning model (i.e., a conveyor belt control decision model). Figure 5 This is a schematic diagram of a conveyor belt control decision model according to an embodiment of the conveyor belt control method of this application. Figure 5 As shown, the State(s) output by the congestion perception module is input into the constructed reinforcement learning model (i.e., the conveyor belt control decision model) into the MLP (Multilayer Perceptron (MLP) is a feedforward artificial neural network model that can map multiple input datasets to a single output dataset), and then outputs the corresponding State(s). The goal is to maximize long-term benefits and then select the action with the highest potential return to execute.

[0109] exist Figure 5 middle, These represent: maintaining the status quo, turning on one conveyor belt, and turning off one conveyor belt. Q is the representation of the Deep Q-Learning algorithm, meaning the long-term benefit of performing the corresponding Action given a state (S). For example, Q(s, The meaning of ) is the long-term benefit of maintaining the status quo given a state State(s); Q(s, The meaning of ) is the long-term benefit of starting a conveyor belt after given a state State(s); Q(s, The meaning of ) is the long-term benefit of shutting down a conveyor belt after a given state (State(s)).

[0110] The training of the reinforcement learning model (i.e., the conveyor belt control decision model) is divided into two parts: offline and online. The offline training process involves acquiring data from past operation records (including video of the monitoring area and records of conveyor belt control), extracting the corresponding State, Action, and Reward, and training the model accordingly. The online training process involves using the offline-trained model to make decisions online and collect data, continuously training and optimizing the model based on the collected data.

[0111] In practical use, this solution's process includes three stages: equipment deployment, model preparation, and field operation, as detailed below:

[0112] 1. Equipment Deployment:

[0113] a) According to Figure 3 As shown, a key monitoring area (i.e., the target area, for example, area A) is designated on the target conveyor belt, and monitoring cameras are installed thereon.

[0114] b) Select the incoming conveyor belt that needs to be controlled, for example, Figure 3 One or more of the main line, branch line, and U-branch line of the conveyor belt.

[0115] 2. Model preparation:

[0116] a) Collect conveyor belt operation data and mark the video clips (i.e., monitoring data) for congestion;

[0117] b) Use the data collected in a) to train the relevant model for the congestion perception module;

[0118] c) Collect relevant data on the linkage between conveyor belt control and monitoring areas;

[0119] d) Use the data collected in c) to train a reinforcement learning model (i.e., train a conveyor belt control decision model).

[0120] 3. On-site operation:

[0121] a) Deploy all relevant models of the congestion perception module and conveyor belt control module in real-world scenarios;

[0122] b) The congestion perception module maps the video clips (i.e., monitoring data) collected by the surveillance cameras to the State (i.e., the current cargo delivery status in the target area).

[0123] c) Input the State into the reinforcement learning model of the conveyor belt control module (i.e. train the conveyor belt control decision model) and calculate the long-term benefit for each Action.

[0124] d) Execute the Action corresponding to the maximum long-term return.

[0125] This application proposes a linkage framework between conveyor belt congestion perception and PLC-based conveyor belt control, which helps control conveyor belt speed, alleviate congestion, and ensure timely delivery. A reinforcement learning algorithm is designed to make decisions on control commands for the conveyor belt based on video footage captured by a monitoring camera, thereby achieving the control objectives. During the design of the reinforcement learning algorithm, the Action output format is optimized to ensure compatibility with different conveyor belt structures.

[0126] Figure 6 This is a schematic diagram of the main units of a conveyor belt control device according to an embodiment of this application. Figure 6 As shown, the conveyor belt control device 600 includes a monitoring data acquisition unit 601, a congestion sensing unit 602, a long-term benefit determination unit 603, and a conveyor belt control unit 604.

[0127] The monitoring data acquisition unit 601 is configured to acquire monitoring data obtained by monitoring the target area of ​​the target conveyor belt.

[0128] The congestion sensing unit 602 is configured to sense congestion in monitoring data in order to determine the current cargo delivery status in the target area.

[0129] The long-term benefit determination unit 603 is configured to input the cargo conveying state into the conveyor belt control decision model to obtain the long-term benefits resulting from performing preset conveyor belt control actions under the cargo conveying state.

[0130] The conveyor belt control unit 604 is configured to determine the target action for conveyor belt control based on long-term benefits and execute the target action for conveyor belt control.

[0131] In some embodiments, the congestion sensing unit 602 is further configured to: extract features from each frame of the monitoring data to obtain a feature sequence corresponding to each frame; input the feature sequence corresponding to each frame into an encoding model to obtain an encrypted feature sequence that combines information from previous and subsequent frames; splice the encrypted feature sequence corresponding to each frame according to the time sequence to obtain a spliced ​​encrypted feature sequence; and determine the spliced ​​encrypted feature sequence as the current cargo delivery status of the target area.

[0132] In some embodiments, the preset conveyor belt control actions include maintaining the status quo, starting a conveyor belt, and closing a conveyor belt; the long-term benefit determination unit 603 is further configured to: input the cargo conveying state into the conveyor belt control decision model to obtain the long-term benefits of maintaining the status quo, starting a conveyor belt, and closing a conveyor belt as output by the conveyor belt control decision model in the cargo conveying state.

[0133] In some embodiments, the conveyor belt control unit 604 is further configured to: determine the preset conveyor belt control action corresponding to the maximum long-term benefit as the conveyor belt control target action; determine the conveyor belt entering the target area corresponding to the conveyor belt control target action; and execute the conveyor belt control target action on the conveyor belt.

[0134] In some embodiments, the conveyor belt control device further includes Figure 6 The model training unit (not shown) is configured to: set the sample cargo transport status of the sample target area of ​​the sample target conveyor belt; set the sample conveyor belt control actions performed under the sample cargo transport status: maintain the status quo, open one conveyor belt, and close one conveyor belt; set the sample long-term benefit, which is collected based on the following method: analyzing historical sample target conveyor belt operation records, adding a preset first negative benefit for each closed conveyor belt; adding a corresponding positive benefit for each opened conveyor belt; if maintaining the status quo or opening one conveyor belt is adopted, then if congestion occurs within a preset number of time units after the conveyor belt control action occurs, adding a preset second negative benefit, and the sum of the final positive and negative benefits is taken as the sample long-term benefit; based on the above settings, a conveyor belt control decision model is trained by the sample cargo transport status, sample conveyor belt control actions, and the sample long-term benefit brought about by performing sample conveyor belt control actions under a given sample cargo transport status.

[0135] In some embodiments, the model training unit is further configured to: determine the corresponding priority of all conveyor belts that receive samples into the target region according to the timeliness requirements; the model training unit is further configured to: turn on the conveyor belt with the highest priority; and the model training unit is further configured to: turn off the conveyor belt with the lowest priority.

[0136] It should be noted that the conveyor belt control method and conveyor belt control device of this application are related in terms of specific implementation, so repeated content will not be described again.

[0137] Figure 7 An exemplary system architecture 700 is shown that can be applied to the conveyor belt control method or conveyor belt control device of the present application embodiments.

[0138] like Figure 7 As shown, system architecture 700 may include terminal devices 701, 702, and 703, a network 704, and a server 705. Network 704 serves as the medium for providing communication links between terminal devices 701, 702, and 703 and server 705. Network 704 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0139] Users can use terminal devices 701, 702, and 703 to interact with server 705 via network 704 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 701, 702, and 703, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0140] Terminal devices 701, 702, and 703 can be various electronic devices with a conveyor belt control and processing screen that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0141] Server 705 can be a server providing various services, such as a backend management server supporting monitoring data obtained by users using terminal devices 701, 702, and 703 to monitor the target area of ​​the conveyor belt via surveillance cameras (this is just an example). The backend management server can acquire monitoring data from the target area of ​​the conveyor belt; perform congestion detection on the monitoring data to determine the current cargo transport status of the target area; input the cargo transport status into the conveyor belt control decision model to obtain the long-term benefits of executing preset conveyor belt control actions under the cargo transport status; determine the conveyor belt control target action based on the long-term benefits, and execute the conveyor belt control target action. Therefore, after identifying congestion, adjusting the conveyor belt's operating status by executing the determined conveyor belt control target action can effectively avoid the dangers that easily arise when manually handling conveyor belt congestion, and can solve the conveyor belt congestion problem at its root, improving the efficiency and accuracy of conveyor belt control.

[0142] It should be noted that the conveyor belt control method provided in this application embodiment is generally executed by server 705, and correspondingly, the conveyor belt control device is generally set in server 705.

[0143] It should be understood that Figure 7 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0144] The following is for reference. Figure 8 It shows a schematic diagram of the structure of a computer system 800 suitable for implementing a terminal device according to the embodiments of this application. Figure 8 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0145] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 802 or programs loaded from storage section 808 into random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the computer system 800. The CPU 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0146] The following components are connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 810 as needed so that computer programs read from it can be installed into storage section 808 as needed.

[0147] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by central processing unit (CPU) 801, it performs the functions defined above in the system of this application.

[0148] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0150] The units described in the embodiments of this application can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a monitoring data acquisition unit, a congestion sensing unit, a long-term benefit determination unit, and a conveyor belt control unit. The names of these units do not necessarily limit the specific unit itself.

[0151] In another aspect, this application also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to acquire monitoring data obtained by monitoring the target area of ​​the target conveyor belt; perform congestion perception on the monitoring data to determine the current cargo conveying status of the target area; input the cargo conveying status into the conveyor belt control decision model to obtain the long-term benefits of executing a preset conveyor belt control action under the cargo conveying status; determine the conveyor belt control target action based on the long-term benefits, and execute the conveyor belt control target action.

[0152] The computer program product of this application includes a computer program that, when executed by a processor, implements the conveyor belt control method in the embodiments of this application.

[0153] According to the technical solution of this application embodiment, after identifying congestion, the operating state of the conveyor belt is adjusted by executing the determined conveyor belt control target action. This can effectively avoid the dangers that are easy to occur when manually handling conveyor belt congestion, and can solve the conveyor belt congestion problem from the root, thereby improving the efficiency and accuracy of conveyor belt control.

[0154] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A conveyor belt control method, characterized in that, include: Acquire monitoring data obtained from monitoring the target area of ​​the target conveyor belt; The monitoring data is used to detect congestion and determine the current cargo delivery status in the target area. The cargo conveying state is input into the conveyor belt control decision model to obtain the long-term benefits of performing preset conveyor belt control actions under the cargo conveying state. The target action for conveyor belt control is determined based on the long-term benefits, and the target action for conveyor belt control is executed.

2. The method according to claim 1, characterized in that, The process of performing congestion sensing on the monitoring data to determine the cargo delivery status corresponding to the monitoring data includes: Feature extraction is performed on each frame of the monitoring data to obtain the feature sequence corresponding to each frame; The feature sequence corresponding to each frame is input into the encoding model to obtain an encrypted feature sequence that combines the information of the preceding and following frames. The encrypted feature sequence corresponding to each frame is concatenated according to the time sequence to obtain the concatenated encrypted feature sequence; The concatenated encrypted feature sequence is determined as the current cargo delivery status of the target area.

3. The method according to claim 1, characterized in that, The preset conveyor belt control actions include maintaining the status quo, starting one conveyor belt, and stopping one conveyor belt; The step of inputting the cargo conveying state into the conveyor belt control decision model to obtain the long-term benefits of executing preset conveyor belt control actions under the cargo conveying state includes: The cargo transport status is input into the conveyor belt control decision model to obtain the long-term benefits of maintaining the status quo, starting a conveyor belt, and shutting down a conveyor belt, as output by the conveyor belt control decision model under the cargo transport status.

4. The method according to claim 1, characterized in that, The step of determining the conveyor belt control target action based on the long-term benefits and executing the conveyor belt control target action includes: The preset conveyor belt control action corresponding to the greatest long-term benefit is determined as the conveyor belt control target action; Determine the conveyor belt that enters the target area corresponding to the conveyor belt control target action, and execute the conveyor belt control target action on the conveyor belt.

5. The method according to claim 1, characterized in that, The method further includes: The sample cargo transport status is set in the sample target area of ​​the sample target conveyor belt that has collected samples; The system is configured to collect and execute sample conveyor belt control actions during the sample cargo transport process: maintain the status quo, start one conveyor belt, and shut down one conveyor belt; A long-term benefit for the sample is set up, which is collected based on the following method: analyzing the historical operation records of the target conveyor belts of the sample, adding a preset first negative benefit for each conveyor belt closed; adding a corresponding positive benefit for each conveyor belt opened; if the status quo is maintained or a conveyor belt is opened, then if congestion occurs within a preset number of time units after the conveyor belt control action, adding a preset second negative benefit; the sum of the final positive and negative benefits is taken as the long-term benefit of the sample. Based on the above settings, a conveyor belt control decision model is obtained by training the sample cargo transport state, the sample conveyor belt control actions, and the long-term benefits of executing the sample conveyor belt control actions under a given sample cargo transport state.

6. The method according to claim 5, characterized in that, The method further includes: Based on the timeliness requirements, determine the corresponding priority for all conveyor belts that converge into the target area of ​​the sample; Starting a conveyor belt includes: starting the conveyor belt with the highest priority; The shutdown of a conveyor belt includes: shutting down the conveyor belt with the lowest priority.

7. A conveyor belt control device, characterized in that, include: The monitoring data acquisition unit is configured to acquire monitoring data obtained by monitoring the target area of ​​the target conveyor belt; A congestion sensing unit is configured to perform congestion sensing on the monitoring data in order to determine the current cargo delivery status of the target area. The long-term benefit determination unit is configured to input the cargo conveying state into the conveyor belt control decision model to obtain the long-term benefits resulting from performing a preset conveyor belt control action in the cargo conveying state. The conveyor belt control unit is configured to determine the conveyor belt control target action based on the long-term benefits and execute the conveyor belt control target action.

8. A conveyor belt control electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images