Human-machine collaborative management and control method, system and device involving logistics storage and transportation

Abstract

The present invention relates to the technical field of intelligent logistics management and control. Disclosed are a human-machine collaborative management and control method, system and device involving logistics storage and transportation. The method comprises: on the basis of a logistics storage and transportation task, determining a target operation process and a corresponding target operation item queue; executing said task on the basis of the queue; determining whether the queue belongs to a manual operation item set; if the queue belongs to the manual operation item set, generating and issuing a manual operation notification instruction; determining whether a confirmation signal fed back on the basis of the manual operation notification instruction is received; if yes, updating the state of operation items to 'in progress'; determining whether the queue belongs to a feedback operation item set; if the queue belongs to the feedback operation item set, determining whether a manual feedback completion signal is received; if yes, updating the state of the operation items to 'completed'; and when the label of one of the operation items is the maximum label in the queue, confirming that said task is completed. According to the present invention, an instruction is sent to an operator and a manual feedback signal is received in time, so as to collaboratively manage and control the logistics storage and transportation operation process requiring manual participation.

Description

Human-machine collaborative control method, system and equipment related to logistics storage and transportation Technical Field

[0001] The present invention relates to the field of intelligent logistics management and control technology, and in particular to a human-machine collaborative management and control method, system and equipment for logistics storage and transportation. Background Art

[0002] The steel industry uses large quantities of raw materials in its daily operations, with steel mills receiving large quantities and a wide variety of raw materials daily. In steel mills, where transportation is primarily based on vehicles, each logistics and transportation process relies on a combination of manual labor and equipment. However, this manual process struggles to coordinate operations in a timely and accurate manner, resulting in a low level of intelligent automation.

[0003] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art.

[0004] Summary of the Invention

[0005] The main purpose of the present invention is to provide a human-machine collaborative control method, system and equipment involving logistics storage and transportation, aiming to solve the technical problem that the manual operation process in the existing steel industry logistics and transportation process is difficult to coordinate as a whole in a timely and accurate manner.

[0006] To achieve the above objectives, the present invention provides a human-machine collaborative management and control method for logistics storage and transportation, comprising:

[0007] Upon receiving a logistics storage and transportation task, determining a target operation process according to the logistics storage and transportation task;

[0008] Determine the target operation item queue corresponding to the target operation process according to the operation item queue;

[0009] Execute the logistics storage and transportation task according to the operation items and operation sequence of the target operation item queue;

[0010] Determine whether the target operation item queue belongs to a manual operation item set;

[0011] When the target operation item queue belongs to the manual operation item set, generating and issuing a manual operation notification instruction;

[0012] Determining whether a confirmation signal based on the manual operation notification instruction feedback is received;

[0013] Upon receiving the confirmation signal, updating the status of the operation item in the target operation item queue to being executed;

[0014] Determining whether the target job item queue belongs to a feedback job item set;

[0015] When the target operation item queue belongs to the feedback operation item set, determining whether a manual feedback completion signal is received;

[0016] Upon receiving the manual feedback completion signal, updating the status of the work item in the target work item queue to completed;

[0017] When the numerical label of the operation item is the maximum numerical label of the target operation item queue, it is confirmed that the logistics storage and transportation task is completed.

[0018] In some embodiments, after determining whether the target work item queue belongs to a manual work item set, the method further includes:

[0019] When the target operation item queue does not belong to the manual operation item set, the corresponding execution equipment is automatically notified through the industrial Internet of Things to start the operation;

[0020] or,

[0021] When the target operation item queue does not belong to the manual operation item set, the operation status corresponding to the operation item that automatically triggers the target operation item queue is in execution, completed, or fault state.

[0022] In some embodiments, after determining whether the target job item queue belongs to the feedback job item set, the method further includes:

[0023] When the target operation item queue does not belong to the feedback operation item set, determining whether an automated feedback completion signal is received;

[0024] When the automation feedback completion signal is received, the status of the work item in the target work item queue is updated to completed.

[0025] In some embodiments, when the target work item queue belongs to the manual work item set, generating and issuing a manual work notification instruction includes:

[0026] When the target operation item queue belongs to the manual operation item set, determining a current operation item of the target operation item queue;

[0027] Obtaining the name audio data and status audio data corresponding to the current operation item;

[0028] Obtaining a job instruction keyword corresponding to the current job item;

[0029] Automatically generate a manual operation notification instruction based on the name audio data and the status audio data in combination with the operation instruction keyword;

[0030] The manual operation notification instruction is issued through a mobile terminal.

[0031] In some embodiments, the sending of the manual operation notification instruction through the mobile terminal includes:

[0032] Determine the operator corresponding to the current operation project based on the preset mapping relationship;

[0033] Obtaining information data of the operator, and searching for the mobile terminal number of the operator according to the information data;

[0034] Establishing a communication connection with the operator based on the mobile terminal number via the mobile terminal;

[0035] When the communication connection is established, the manual operation notification instruction is played by voice and displayed on a preset display terminal.

[0036] In some embodiments, the method further comprises:

[0037] Determine the operation items and the operation sequence of the operation items according to the preset production control items;

[0038] Classifying the work tasks based on the work items and work sequences to generate a set of work process types;

[0039] A job item queue is constructed according to the job flow type set.

[0040] In some embodiments, constructing a job item queue according to the job flow type set includes:

[0041] Extracting a plurality of operation processes and the operation items, operation sequence and initial conditions of each operation process according to the operation process type set;

[0042] Constructing a work task according to the work items, work sequence and initial conditions of the work process; wherein each work task includes a plurality of work items;

[0043] A job item queue is constructed according to the job tasks.

[0044] In some embodiments, the method further comprises:

[0045] Determine the operation items according to the preset production control items;

[0046] The work items are classified according to their task execution attributes to generate a set of manual work items and a set of feedback work items.

[0047] In addition, to achieve the above-mentioned purpose, the present invention also proposes a human-machine collaborative management and control system involving logistics storage and transportation, comprising:

[0048] The operation task process control module is used to determine the target operation process according to the logistics storage and transportation task when receiving the logistics storage and transportation task;

[0049] The operation task process control module is further used to determine the target operation project queue corresponding to the target operation process according to the operation project queue;

[0050] The operation task process control module is further used to execute the logistics storage and transportation tasks according to the operation items and operation sequence of the target operation item queue;

[0051] The job task process control module is also used to determine whether the target job item queue belongs to the manual job item set;

[0052] The operation task process control module is further configured to generate and issue a manual operation notification instruction when the target operation item queue belongs to the manual operation item set;

[0053] The operation task process control module is further used to determine whether a confirmation signal based on the manual operation notification instruction feedback is received;

[0054] a task status determination module, configured to update the status of the task item in the target task item queue to being executed upon receiving the confirmation signal;

[0055] The job task process control module is further used to determine whether the target job item queue belongs to the feedback job item set;

[0056] The operation task process control module is further configured to determine whether a manual feedback completion signal is received when the target operation item queue belongs to the feedback operation item set;

[0057] The task status determination module is further configured to update the status of the work item in the target work item queue to completed upon receiving the manual feedback completion signal;

[0058] The operation task process control module is also used to confirm that the logistics storage and transportation task is completed when the digital label of the operation project is the maximum digital label of the target operation project queue.

[0059] In addition, to achieve the above-mentioned purpose, the present invention also proposes a human-machine collaborative control device involving logistics storage and transportation, which includes: a memory, a processor, and a human-machine collaborative control program involving logistics storage and transportation stored on the memory and runnable on the processor, and the human-machine collaborative control program involving logistics storage and transportation is configured to implement the human-machine collaborative control method involving logistics storage and transportation as described above.

[0060] The present invention provides a human-machine collaborative management and control method for logistics storage and transportation, comprising: upon receiving a logistics storage and transportation task, determining a target operation process according to the logistics storage and transportation task; determining a target operation project queue corresponding to the target operation process according to an operation project queue; executing the logistics storage and transportation task according to the operation items and operation sequence of the target operation project queue; judging whether the target operation project queue belongs to a manual operation project set; when the target operation project queue belongs to the manual operation project set, generating and issuing a manual operation notification instruction; judging whether a confirmation signal based on feedback of the manual operation notification instruction is received; upon receiving the confirmation signal, updating the status of the operation items in the target operation project queue to being executed; judging whether the target operation project queue belongs to a feedback operation project set; when the target operation project queue belongs to the feedback operation project set, judging whether a manual feedback completion signal is received; upon receiving the manual feedback completion signal, updating the status of the operation items in the target operation project queue to being completed; and confirming that the logistics storage and transportation task is completed when the digital label of the operation item is the maximum digital label of the target operation item queue. In the present invention, the overall execution process of logistics storage and transportation tasks is intelligently controlled, operation instructions are sent to operators in a timely manner, and manual feedback on the execution status is received, thereby coordinating the management and control of logistics storage and transportation operation processes that require manual participation, solving the technical problem that the operation processes that require manual participation in the existing steel industry logistics and transportation processes are difficult to coordinate in a timely and accurate manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] FIG1 is a schematic diagram of the structure of a human-machine collaborative control device for logistics storage and transportation in a hardware operating environment according to an embodiment of the present invention;

[0062] FIG2 is a flow chart of a first embodiment of a human-machine collaborative management and control method for logistics storage and transportation according to the present invention;

[0063] FIG4 is a diagram illustrating a human-machine collaborative control method for logistics storage and transportation according to the present invention;

[0064] FIG4 is a flow chart of a second embodiment of a method for human-machine collaborative management and control of logistics storage and transportation according to the present invention;

[0065] FIG5 is a structural block diagram of the first embodiment of the human-machine collaborative control device for logistics storage and transportation according to the present invention.

[0066] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0067] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0068] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0069] In addition, the descriptions of "first", "second", etc. in the present invention are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention. It should be understood that the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention.

[0070] 1 , which is a schematic diagram of the structure of a human-machine collaborative management and control device involving logistics storage and transportation in a hardware operating environment according to an embodiment of the present invention.

[0071] As shown in Figure 1, the human-machine collaborative management and control equipment involved in logistics storage and transportation may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM memory) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0072] Those skilled in the art will understand that the structure shown in FIG1 does not constitute a limitation on human-machine collaborative control equipment involved in logistics storage and transportation, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0073] As shown in FIG1 , the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a human-machine collaborative management and control program related to logistics storage and transportation.

[0074] In the human-machine collaborative control device involving logistics storage and transportation shown in Figure 1, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the human-machine collaborative control device involving logistics storage and transportation of the present invention can be set in the human-machine collaborative control device involving logistics storage and transportation, and the human-machine collaborative control device involving logistics storage and transportation calls the human-machine collaborative control program involving logistics storage and transportation stored in the memory 1005 through the processor 1001, and executes the human-machine collaborative control method involving logistics storage and transportation provided by the embodiment of the present invention.

[0075] An embodiment of the present invention provides a method for human-machine collaborative control involving logistics storage and transportation. Referring to FIG. 2 , FIG. 2 is a flow chart of a first embodiment of a method for human-machine collaborative control involving logistics storage and transportation according to the present invention.

[0076] As shown in FIG2 , the human-machine collaborative control method for logistics storage and transportation includes:

[0077] Step S10: upon receiving a logistics storage and transportation task, determining a target operation process according to the logistics storage and transportation task;

[0078] Step S20: determining a target operation item queue corresponding to the target operation process according to the operation item queue;

[0079] Step S30: executing the logistics storage and transportation task according to the operation items and operation sequence of the target operation item queue;

[0080] Step S40: determining whether the target operation item queue belongs to a manual operation item set;

[0081] Step S50: When the target operation item queue belongs to the manual operation item set, a manual operation notification instruction is generated and issued;

[0082] Step S60: determining whether a confirmation signal based on the manual operation notification instruction feedback is received;

[0083] Step S70: upon receiving the confirmation signal, updating the status of the operation item in the target operation item queue to being executed;

[0084] Step S80: determining whether the target job item queue belongs to a feedback job item set;

[0085] Step S90: When the target operation item queue belongs to the feedback operation item set, determining whether a manual feedback completion signal is received;

[0086] Step S100: upon receiving the manual feedback completion signal, updating the status of the work item in the target work item queue to completed;

[0087] Step S110: When the digital number of the operation item is the maximum digital number of the target operation item queue, confirm that the logistics storage and transportation task is completed.

[0088] It should be noted that the execution subject in this embodiment may be a human-machine collaborative control device involved in logistics storage and transportation. The human-machine collaborative control device involved in logistics storage and transportation may be a computer device with data processing functions, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment, a computer device is used as an example for explanation.

[0089] For example, this embodiment is described by taking the in-plant storage and transportation operation process of raw material procurement logistics of a steel enterprise as an example, and the transportation method is illustratively automobile transportation.

[0090] In one embodiment, the work items and the work sequence of the work items are determined according to the preset production control items; the work tasks are classified based on the work items and the work sequence to generate a set of work process types; and the work item queue is constructed according to the set of work process types.

[0091] Specifically, establish a set of operation process types: classify each operation task according to the operation items and their order in the production control process, and establish an operation process type set C j ; Where 1≤j≤m, m is the total number of types of all operation processes in the production control process.

[0092] In one embodiment, a job item queue is constructed based on the job process type set, including: extracting several job processes and the job items, job sequence and initial conditions of each job process based on the job process type set; constructing job tasks based on the job items, job sequence and initial conditions of the job processes; wherein each job task includes several job items; and constructing a job item queue based on the job tasks.

[0093] For example, a job item queue corresponding to each job flow type is established: for each type of job flow C j (1≤j≤m) Establish a unique description standard, including all the work items included in the process, the order of execution of each project and the initial conditions. For example, task C j Contains n job items, forming a project queue C ji ; Among them, 1≤j≤m, 1≤i≤n.

[0094] In one embodiment, work items are determined based on preset production control items; and the work items are classified according to task execution attributes to generate a manual work item set and a feedback work item set.

[0095] For example, each work project is classified to establish a set T of manual work projects that need to wait for system notification before starting, and a set Y of feedback work projects that need to wait for manual feedback before confirmation of completion. U is the set of all work projects.

[0096] Specifically, a set of manual work items T, T∈U, that need to wait for system notification before starting is established. For the work items T in the set T i ; Where 1≤i≤k, assuming that the set T contains k work items. The work items T in the set T iWhen triggered to the "executable" state, the "Start Task" instruction can be automatically sent to on-site operators through the interactive application module, notifying the corresponding personnel to start the task and receiving feedback. Job instructions (such as voice, character, or graphic job instructions) are sent through the mobile terminal, confirmation replies are received, and the job status is automatically updated. After confirming that the instruction has been successfully sent, according to the state machine of the current task, T i 's status is updated to "In Execution".

[0097] Specifically, a set of job items that require manual feedback to confirm completion is established, namely the feedback job item set Y, where Y ∈ U. For the job item Y in the set Y i ; where 1 ≤ i ≤ j, assuming that the set Y contains a total of j jobs. The job item Y in the set Y i has been completed, the on-site operator wakes up the mobile terminal through a voice command or a button press, enters the feedback information, and the feedback information can be received through the interactive application module. For example, through the mobile terminal, the feedback information from the on-site operator (such as voice and characters) is received, the job status is automatically updated, and after confirming that the feedback information has been successfully recognized, according to the state machine of the current task, Y i 's status is updated to "Completed".

[0098] It should be noted that the status of a specific task can be adjusted through the job task state machine: for a specific job task W, according to its corresponding job process type C j (1 ≤ j ≤ m), the corresponding job item queue C ji (1 ≤ j ≤ m, 1 ≤ i ≤ n) can be determined. For each item in the job item queue C ji , five statuses of "To Be Executed, Executable, In Execution, Completed, Abnormal" are set.

[0099] In practical applications, when the process starts, that is, when C j1 meets the execution conditions, C j1 is triggered to the "executable" state, and the status of C ji (1 < i ≤ n) is set to "To Be Executed". For C ji , (1 < i ≤ n): when it is sensed that Cji starts to execute, Cji is triggered to the "In Execution" state; when it is sensed that C ji is "Completed", C ji is triggered to the "Completed" state; when C ji (i < n) is in the "Completed" state, C j(i+1) is triggered to the "executable" state; when C ji (i = n) is in the "Completed" state, the item status is set to "Completed". When C jiIf the trigger conditions of the above-mentioned "to be executed, executable, executing, completed" states are met, but the corresponding state is not entered within the set time, or the time in the "executable" or "executing" state exceeds the pre-set time, C is triggered. ji For C ji , when i=n, when C ji When the status is "Completed", the current project is triggered to be completed.

[0100] It is understandable that establishing a multi-modal interactive information support dataset such as voice, pictures, and text for the information that humans need to interact with during the execution of tasks, that is, establishing a multi-modal human-computer interaction support dataset, is conducive to effectively improving the recognition accuracy and interaction efficiency of human feedback information.

[0101] In practical applications, establish a multimodal human-computer interaction support dataset: For task C ji (1≤i≤n), establish a unique job name instruction audio dataset N; establish a status instruction audio dataset S corresponding to the five job states of "to be executed, executable, executing, completed, and abnormal"; establish a feedback dataset, including a voice feedback keyword audio training dataset Ra, a picture feedback training dataset Rp, and a gesture feedback instruction dataset Rg; establish a voice query keyword audio dataset F; establish a job instruction keyword audio dataset O; establish an executor information set P corresponding to each job (including the dial-up number of the mobile terminal device); establish a mapping relationship dataset E between the operator and each specific job project to achieve each job C i and its operator P i (P i It should be noted that the specific content of the multimodal human-computer interaction support dataset includes but is not limited to the above set, and can be set according to actual conditions.

[0102] It can be understood that establishing an interactive information conversion module can achieve efficient and highly reliable conversion between text and voice for the data contained in the aforementioned sets N, S, Ra, Rp, Rg, F, O, P, and E. Character information can be converted into audio data from the aforementioned sets to form voice commands, which can be played to on-site workers. Automatic recognition of voice commands can also be used to convert audio information into character information for operational process control. Feedback images and gesture information can also be converted into character information for operational process control.

[0103] In one embodiment, when a logistics storage and transportation task is received, a target operation process is determined according to the logistics storage and transportation task, a target operation item queue corresponding to the target operation process is determined according to the operation item queue, and the logistics storage and transportation task is executed according to the operation items and operation sequence of the target operation item queue: for a specific operation task, for example, a vehicle driver receives an operation and maintenance task while waiting in a queue, the operation process C uniquely corresponding to the operation and maintenance task is determined. j For a specific job task, determine its unique corresponding job item queue C ji , initialize the variable i to 1. Execute queue C ji The i-th homework item in C ji The status is updated to "Executable".

[0104] In one embodiment, it is determined whether the target operation item queue belongs to a manual operation item set; if the target operation item queue belongs to the manual operation item set, a manual operation notification instruction is generated and issued; it is determined whether a confirmation signal based on the feedback of the manual operation notification instruction is received; if the confirmation signal is received, the status of the operation item in the target operation item queue is updated to being executed; and it is determined that C ji Does it belong to the set T (a set of manual tasks that need to wait for system notification before starting), if C ji Belong to the set T, through the interactive control information conversion module and the multimodal interactive application module, the operation instruction is issued to the corresponding operator. If the corresponding operator feedback confirmation information is not received, repeat the steps to issue the operation instruction to the corresponding operator through the interactive control information conversion module and the multimodal interactive application module; if the reception confirmation information feedback from the operator is still not received after the specified time, an abnormal prompt is issued; if the reception confirmation information feedback from the operator is received, C ji The status is updated to "Executing".

[0105] In one embodiment, when the target operation project queue belongs to the manual operation project set, a manual operation notification instruction is generated and issued, including: when the target operation project queue belongs to the manual operation project set, determining the current operation project of the target operation project queue; obtaining the name audio data and status audio data corresponding to the current operation project; obtaining the operation instruction keyword corresponding to the current operation project; automatically generating a manual operation notification instruction based on the name audio data and status audio data combined with the operation instruction keyword; and issuing the manual operation notification instruction through a mobile terminal.

[0106] Specifically, C ji Belongs to set T, executes queue C jiThe i-th job item in the set T, the job item T in the set T i When triggered to the "executable" state, the interactive control information conversion module and the multimodal interactive application module can automatically issue a "start operation" instruction to the on-site workers, notify the corresponding personnel to start the operation, and receive feedback. Send operation instructions (such as voice, character or graphic type operation instructions) through the mobile terminal, receive confirmation replies, and automatically update the operation status. After confirming that the instruction is successfully issued, according to the state machine of the current task, T i The status is updated to "Executing".

[0107] In another embodiment, after determining whether the target operation project queue belongs to the manual operation project set, it also includes: when the target operation project queue does not belong to the manual operation project set, automatically notifying the corresponding execution equipment through the industrial Internet of Things to start the operation; or, when the target operation project queue does not belong to the manual operation project set, automatically triggering the operation status corresponding to the operation project of the target operation project queue to be in execution, completed or fault state.

[0108] It can be understood that, a set C of all job items other than the set T and the set Y is established, that is, when C does not belong to (T∪Y), for the job C in the set C i ; Where 1≤i≤n, assuming that set C contains n jobs. According to the state machine of the current task, when C is triggered i When the state is "executable", the corresponding execution equipment can be automatically notified through the industrial Internet of Things to start the operation; the C i The status is "In Progress", "Completed", or "Failed".

[0109] Specifically, if C ji Does not belong to the set T, send instructions to the corresponding execution equipment to start automatic operation, and judge whether the operation has started based on the automatically collected basic automation information. If the operation has started, C ji The status is updated to "Executing"; if the signal that the job has started is not received within the specified time, an abnormal prompt will be issued.

[0110] In one embodiment, it is determined whether the target job item queue belongs to the feedback job item set; when the target job item queue belongs to the feedback job item set, it is determined whether a manual feedback completion signal is received; when the manual feedback completion signal is received, the status of the job item in the target job item queue is updated to completed.

[0111] In one embodiment, after determining whether the target job item queue belongs to the feedback job item set, it further includes: when the target job item queue does not belong to the feedback job item set, determining whether an automated feedback completion signal is received; when the automated feedback completion signal is received, updating the status of the job items in the target job item queue to completed.

[0112] Specifically, determine whether C ji belongs to set Y (the feedback job item set Y that requires manual feedback to confirm completion). In one example, if C ji belongs to set Y, determine whether the job completion information with manual feedback is received. If received, update the status of C ji [ to "completed". If not received, continue to wait. If the job completion information from the operator is still not received after the specified time, an exception prompt is issued. In another example, if C ji does not belong to set Y, determine whether the job completion information with automated system feedback is received. If received, update the status of C ji to "completed"; if not received, continue to wait. If the job completion information from the automated system is still not received after the specified time, an exception prompt is issued.

[0113] It should be noted that when the digital label of the job item is the largest digital label of the target job item queue, it is confirmed that the logistics storage and transportation task is completed: if i < n (n is the total number of items in queue C ji ), then i = i + 1, jump to step to execute the i-th job item in queue C ji , and update the status of C ji to "executable"; if i = n (n is the total number of items in queue C ji ), then the task ends.

[0114] This embodiment determines the target operation process according to the logistics storage and transportation task when receiving the logistics storage and transportation task; determines the target operation project queue corresponding to the target operation process according to the operation project queue; executes the logistics storage and transportation task according to the operation items and operation sequence of the target operation project queue; determines whether the target operation project queue belongs to the manual operation project set; generates and issues a manual operation notification instruction when the target operation project queue belongs to the manual operation project set; determines whether a confirmation signal based on the feedback of the manual operation notification instruction is received; when the confirmation signal is received, updates the status of the operation project in the target operation project queue to being executed; determines whether the target operation project queue belongs to the feedback operation project set; when the target operation project queue belongs to the feedback operation project set, determines whether a manual feedback completion signal is received; when the manual feedback completion signal is received, updates the status of the operation project in the target operation project queue to being completed; and confirms that the logistics storage and transportation task is completed when the digital label of the operation project is the maximum digital label of the target operation project queue. In this embodiment, the overall execution process of logistics storage and transportation tasks is intelligently controlled, operation instructions are sent to operators in a timely manner, and manual feedback on the execution status is received, thereby coordinating the management and control of logistics storage and transportation operations that require manual participation, solving the technical problem that the manual participation operations in the existing steel industry logistics and transportation processes are difficult to coordinate in a timely and accurate manner.

[0115] In some embodiments, as shown in FIG3 , a second embodiment of the human-machine collaborative management and control method for logistics storage and transportation according to the present invention is proposed based on the first embodiment. Step S50 includes:

[0116] Step S51: when the target operation item queue belongs to the manual operation item set, determining the current operation item of the target operation item queue;

[0117] Step S52: Acquire the name audio data and status audio data corresponding to the current operation item;

[0118] Step S53: Obtaining the operation instruction keyword corresponding to the current operation item;

[0119] Step S54: Automatically generate a manual operation notification instruction based on the name audio data and the status audio data in combination with the operation instruction keyword;

[0120] Step S55: Send the manual operation notification instruction via the mobile terminal.

[0121] In one embodiment, the manual operation notification instruction is issued through a mobile terminal, including: determining the operator corresponding to the current operation project based on a preset mapping relationship; obtaining the information data of the operator, and searching for the mobile terminal number of the operator based on the information data; establishing a communication connection with the operator based on the mobile terminal number through the mobile terminal; when the communication connection is established, playing the manual operation notification instruction by voice and displaying the manual operation notification instruction on a preset display terminal.

[0122] Specifically, C ji Belongs to set T, executes queue C ji The i-th job item in the set T, the job item T in the set T i When triggered to the "executable" state, the interactive control information conversion module and the multimodal interactive application module can automatically issue a "start operation" instruction to the on-site workers, notify the corresponding personnel to start the operation, and receive feedback. Send operation instructions (such as voice, character or graphic type operation instructions) through the mobile terminal, receive confirmation replies, and automatically update the operation status. After confirming that the instruction is successfully issued, according to the state machine of the current task, T i The status is updated to "Executing".

[0123] It should be noted that a multimodal interactive application module is established to enable interaction between people and the control system through various forms such as voice commands, pictures, and gestures.

[0124] In one example, a mobile terminal sends a job instruction (including voice, characters and graphics), receives a confirmation reply, and automatically updates the job status: corresponding to the current job project C ji , according to the name of the audio data N corresponding to the project i and status audio data S i , combined with the job instruction keyword O i , automatically generates the voice command "O i +N i +S i " and the corresponding character / graphic instructions. The interactive module is based on the mapping relationship E i , automatically assigning the job item C ji Mapped to the corresponding on-site operator P i According to P i The information data of the mobile terminal is used to find the configured mobile terminal number and automatically dial the number; after the call is connected, the voice command "O i +N i +S i ", and at the same time display the corresponding character / graphic instructions on the mobile terminal screen; finally receive the character or voice confirmation information replied by the on-site operator. Recognize the character or voice reply information: For the voice reply information Ri , automatically recognize the voice reply content by comparing the voice feedback keyword audio dataset R. If the character or voice reply information is recognized as confirmed successfully, C is automatically updated ji Project status, the updated status is displayed on the mobile terminal screen interface i , while playing the updated C ji Status audio information S i If the recognition character or voice reply information is not confirmed successfully, the job instruction will be automatically issued again, and the interactive module will be executed again according to the mapping relationship E i , automatically assigning the job item C ji Mapped to the corresponding on-site operator P i According to P i The information data of the mobile terminal is used to find the configured mobile terminal number and automatically dial the number; after the call is connected, the voice command "O i +N i +S i ", while the corresponding character / graphic instructions are displayed on the mobile terminal screen; finally, the on-site operator responds with character or voice confirmation information and recognizes the character or voice reply information. If the set time and number of times are exceeded and there is still no successful recognition, an abnormal status alarm is triggered.

[0125] For example, in actual applications: Scenario 1: In a logistics management system, vehicle drivers, while waiting in line, can use voice commands played on their in-vehicle mobile terminals to obtain information about the immediate start of unloading operations at the factory, and interact with the logistics management system through feedback confirmation information. Scenario 2: In an equipment operation and maintenance system, operators can use mobile terminals such as head-mounted displays and PADs to play voice commands or display text instructions to obtain information about the immediate start of a specific operation and maintenance task, and interact with the equipment operation and maintenance system through feedback confirmation information.

[0126] In another example, feedback information (including voice and text) from on-site workers is received through a mobile terminal to automatically update the operation status: Step 1: The on-site worker wakes up the mobile terminal through voice commands or keystrokes, and the mobile terminal sends a request to the interactive application module to obtain the corresponding operation project information of the worker. Step 2: After receiving the request from the mobile terminal, the interactive application module first verifies the identity of the worker through the mobile terminal number and login information. i , the interaction module is based on the mapping relationship E i , get P i Corresponding current job project C ji and C jiCurrent status, and send relevant information to the corresponding mobile terminal. Step 3. On-site workers can view the current corresponding work items and status through the mobile terminal, and input the feedback voice information according to the feedback instruction keyword; or input the feedback information through the corresponding interface of the mobile terminal; or feedback the captured video or picture; or use gesture feedback. Then the feedback information is sent back to the interactive application module. Step 4. Identify and confirm the feedback information. According to the feedback data set Ra, Rp, Rg, the voice instruction, image picture, and gesture instruction are recognized. If the recognition is successful, it is combined with C ji Current status, update C based on feedback content ji The operation status is displayed on the mobile terminal screen interface, and the updated status is played at the same time. ji If the recognition fails in step 3 and the on-site operator finds that the status is not updated normally, they can provide feedback again. After receiving the feedback, the system will repeat "steps 2" and "steps 3". If the recognition fails after the set time and number of times, an abnormal status alarm will be triggered.

[0127] For example, in actual applications: Scenario 1: In a logistics management system, when a vehicle arrives at a manual work location such as unloading or vehicle inspection, on-site workers can wake up terminals such as head-mounted displays (HMDs) or PADs through voice commands or button presses. The terminal displays information such as the worker's corresponding work item and status. After confirming that everything is correct, the worker uses gestures or voice to guide the camera on the HMD to take photos of specified areas such as the license plate and vehicle compartment. The worker confirms the photos through voice or gestures and sends the photos back to the interactive application module. Alternatively, the worker can input feedback information such as "unloading completed" or "vehicle inspection results" through voice, and after confirmation, send the feedback information back to the interactive application module. Scenario 2: In an equipment operation and maintenance system, when the operation and maintenance personnel are performing operations such as part replacement on-site, they can wake up terminals such as HMDs and PADs through voice commands or button presses. The terminal displays the worker's corresponding work item and status information. After confirming that everything is correct, the worker uses gestures or voice to direct the camera on the head-mounted display to take photos of the equipment being maintained, the spare parts to be replaced, the key steps in replacing the equipment, and the damaged parts that have been replaced. The worker confirms the photos with voice or gestures, and then transmits the photos back to the interactive application module. Alternatively, the worker can wake up the head-mounted display with a voice command, input feedback information such as "replacement completed" through voice, and then confirm the feedback information and transmit it back to the interactive application module.

[0128] In another example, a mobile terminal receives a query (including voice and text) from a field operator and responds: Step 1: The field operator wakes up the mobile terminal through voice commands or keystrokes, and inputs a voice query message or key text referring to the query keyword. Step 2: After receiving the query message, the interactive application module verifies the operator's identity using the mobile terminal number and login information. i , according to the interactive module according to the mapping relationship E i , get P i Corresponding information. Step 3. Identify the query information. For voice query information, automatically identify the voice query content by comparing it with the voice query keyword audio data set Pa. Step 4. If the recognition is successful, the system retrieves the corresponding information in the database based on the query keyword, and converts the corresponding information into voice information for playback through the mobile terminal, while providing relevant text and graphic displays. Step 5. If the recognition fails, play the audio or display a prompt message asking the operator to re-enter the query information. After receiving the query information, repeat "Step 2", "Step 3" and "Step 4". Step 6. If the set time and number of times are exceeded and the recognition is still unsuccessful, the abnormal status alarm is triggered.

[0129] For example, in actual applications: Scenario 1: In a logistics management system, warehouse managers can wake up mobile terminals such as headsets and PADs through voice commands or buttons, and then enter query instructions for the storage location of a certain type of goods through voice or text. Interactive application modules obtain query results through processes such as identity authentication, voice recognition, and data retrieval, and then transmit the query results to the mobile terminal for playback or display. Scenario 2: In an equipment operation and maintenance system, operation and maintenance personnel can wake up mobile terminals such as headsets and PADs through voice commands or buttons, and then enter query instructions for the current work project through voice or text. Interactive application modules obtain query results through processes such as identity authentication, voice recognition, and data retrieval, and then transmit the query results to the mobile terminal for playback or display.

[0130] This embodiment determines the current work item of the target work item queue when the target work item queue belongs to the manual work item set; obtains the name audio data and status audio data corresponding to the current work item; obtains the work instruction keyword corresponding to the current work item; automatically generates a manual work notification instruction based on the name audio data and status audio data combined with the work instruction keyword; and sends the manual work notification instruction through a mobile terminal. In this embodiment, a multimodal interactive application module is established to interact with the management and control system in various forms such as voice instructions, pictures, and gestures. The overall execution process of logistics storage and transportation tasks is intelligently controlled, and work instructions are sent to operators in a timely manner and manual feedback on the execution status is received, thereby collaboratively controlling the logistics storage and transportation operation process that requires manual participation, solving the technical problem that it is difficult to conduct timely and accurate overall coordination of the manual participation operation process in the existing steel industry logistics and transportation process.

[0131] 4 , which is a structural block diagram of a first embodiment of a human-machine collaborative management and control system for logistics storage and transportation according to the present invention.

[0132] As shown in FIG4 , the human-machine collaborative management and control system for logistics storage and transportation includes:

[0133] The operation task process control module 10 is used to determine the target operation process according to the logistics storage and transportation task when receiving the logistics storage and transportation task;

[0134] The operation task process control module 10 is further configured to determine a target operation project queue corresponding to the target operation process according to the operation project queue;

[0135] The operation task process control module 10 is further used to execute the logistics storage and transportation task according to the operation items and operation sequence of the target operation item queue;

[0136] The operation task process control module 10 is further used to determine whether the target operation project queue belongs to the manual operation project set;

[0137] The operation task process control module 10 is further configured to generate and issue a manual operation notification instruction when the target operation item queue belongs to the manual operation item set;

[0138] The operation task process control module 10 is further used to determine whether a confirmation signal based on the manual operation notification instruction feedback is received;

[0139] The task status determination module 20 is configured to update the status of the task item in the target task item queue to being executed upon receiving the confirmation signal;

[0140] The job task process control module 10 is further configured to determine whether the target job item queue belongs to the feedback job item set;

[0141] The task process control module 10 is further configured to determine whether a manual feedback completion signal is received when the target task queue belongs to the feedback task set;

[0142] The task status determination module 20 is further configured to update the status of the task item in the target task item queue to completed upon receiving the manual feedback completion signal;

[0143] The operation task process control module 10 is further used to confirm that the logistics storage and transportation task is completed when the digital label of the operation project is the maximum digital label of the target operation project queue.

[0144] In this embodiment, the human-machine collaborative control system involving logistics storage and transportation includes an operation task process control module and a task status determination module as an example for explanation. The human-machine collaborative control system involving logistics storage and transportation is used to collaboratively control the (logistics storage and transportation) operation process that requires human participation. The human-machine collaborative control system involving logistics storage and transportation can send operation instructions to the operators in a timely manner through voice or mobile terminal display and receive manual feedback on the execution status, thereby solving the problem that the operation process with human participation is difficult to coordinate as a whole in a timely and accurate manner. It can be understood that the human-machine collaborative control system involving logistics storage and transportation can also include: an operation process control system (including a scheduling model for all waiting personnel in each step), a manual operation status acquisition system, a mobile application system, automatic generation of voice content, a feedback recognition system, and a voice dialing system.

[0145] Exemplarily, referring to FIG5 , a human-machine collaborative management and control system involving logistics storage and transportation may include an operation task process management and control system, an operation task state machine, a multimodal human-machine interaction data set, an interactive control information conversion module, and a multimodal interactive application module.

[0146] Specifically, as shown in Figure 5, the task process control system manages the execution process and status of tasks based on the tasks and their sequence. This primarily involves automatically collecting key system information and issuing task instructions in real time. The task state machine automatically determines and updates the current status of each task based on real-time system information, automatically determining the task that needs to be executed in real time. The multimodal human-computer interaction dataset establishes a multimodal interactive information support dataset, including voice, images, and text, for the information that humans interact with during task execution. This effectively improves the accuracy of human feedback recognition and interaction efficiency. The interactive information conversion module converts system instructions into images, voice, and other information that is easily recognized by human operators. It also recognizes human feedback, such as voice, gestures, and images, and converts the recognition results into usable information for the system. The multimodal interactive application module enables real-time interaction between human operators, production managers, and the control system through various means, including voice, graphics, gestures, and video recording. This primarily includes applications such as issuing instructions, receiving feedback, and querying and issuing task status.

[0147] This embodiment determines the target operation process according to the logistics storage and transportation task when receiving the logistics storage and transportation task; determines the target operation project queue corresponding to the target operation process according to the operation project queue; executes the logistics storage and transportation task according to the operation items and operation sequence of the target operation project queue; determines whether the target operation project queue belongs to the manual operation project set; generates and issues a manual operation notification instruction when the target operation project queue belongs to the manual operation project set; determines whether a confirmation signal based on the feedback of the manual operation notification instruction is received; when the confirmation signal is received, updates the status of the operation project in the target operation project queue to being executed; determines whether the target operation project queue belongs to the feedback operation project set; when the target operation project queue belongs to the feedback operation project set, determines whether a manual feedback completion signal is received; when the manual feedback completion signal is received, updates the status of the operation project in the target operation project queue to being completed; and confirms that the logistics storage and transportation task is completed when the digital label of the operation project is the maximum digital label of the target operation project queue. In this embodiment, the overall execution process of logistics storage and transportation tasks is intelligently controlled, operation instructions are sent to operators in a timely manner, and manual feedback on the execution status is received, thereby coordinating the management and control of logistics storage and transportation operations that require manual participation, solving the technical problem that the manual participation operations in the existing steel industry logistics and transportation processes are difficult to coordinate in a timely and accurate manner.

[0148] In addition, for technical details that are not fully described in the embodiment of the human-machine collaborative control system involving logistics storage and transportation, please refer to the human-machine collaborative control method involving logistics storage and transportation as described above provided in any embodiment of the present invention, and will not be repeated here.

[0149] It should be understood that the above is only an example and does not constitute any limitation to the technical solution of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any limitation on this.

[0150] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the embodiment scheme, and no limitation is made here.

[0151] In addition, it should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0152] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0153] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, or of course by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0154] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A human-machine collaborative control method for logistics storage and transportation, characterized in that: The human-machine collaborative control method involving logistics storage and transportation includes: When receiving a logistics storage and transportation task, determining a target operation process according to the logistics storage and transportation task; Determine the target operation item queue corresponding to the target operation process according to the operation item queue; Execute the logistics storage and transportation task according to the operation items and operation sequence of the target operation item queue; Determine whether the target operation item queue belongs to a manual operation item set; When the target operation item queue belongs to the manual operation item set, generating and issuing a manual operation notification instruction; Determining whether a confirmation signal based on the manual operation notification instruction feedback is received; When the confirmation signal is received, updating the status of the operation item in the target operation item queue to being executed; Determine whether the target job item queue belongs to a feedback job item set; When the target operation item queue belongs to the feedback operation item set, determining whether a manual feedback completion signal is received; When the manual feedback completion signal is received, the status of the work item in the target work item queue is updated to be completed; When the digital number of the operation item is the maximum digital number of the target operation item queue, it is confirmed that the logistics storage and transportation task is completed.

2. The human-machine collaborative control method involving logistics storage and transportation as claimed in claim 1 is characterized in that: After determining whether the target operation item queue belongs to a manual operation item set, the method further includes: When the target operation item queue does not belong to the manual operation item set, the corresponding execution equipment is automatically notified to start the operation through the industrial Internet of Things; or, When the target operation item queue does not belong to the manual operation item set, the operation status corresponding to the operation item that automatically triggers the target operation item queue is in execution, completed or faulty.

3. The human-machine collaborative control method involving logistics storage and transportation as claimed in claim 1 is characterized in that: After determining whether the target job item queue belongs to the feedback job item set, the method further includes: When the target operation item queue does not belong to the feedback operation item set, determining whether an automated feedback completion signal is received; When the automation feedback completion signal is received, the status of the operation item in the target operation item queue is updated to be completed.

4. The human-machine collaborative control method involving logistics storage and transportation as claimed in claim 1 is characterized in that: When the target operation item queue belongs to the manual operation item set, generating and issuing a manual operation notification instruction includes: When the target operation item queue belongs to the manual operation item set, determining a current operation item of the target operation item queue; Obtain name audio data and status audio data corresponding to the current job item; Obtaining a job instruction keyword corresponding to the current job item; Automatically generate a manual operation notification instruction based on the name audio data and the status audio data combined with the operation instruction keyword; The manual operation notification instruction is issued through a mobile terminal.

5. The human-machine collaborative control method involving logistics storage and transportation as claimed in claim 4 is characterized in that: The sending of the manual operation notification instruction through the mobile terminal includes: Determine the operator corresponding to the current operation project based on the preset mapping relationship; Acquire information data of the operator, and search for the mobile terminal number of the operator according to the information data; Establishing a communication connection with the operator through a mobile terminal based on the mobile terminal number; When the communication connection is established, the manual operation notification instruction is played by voice and displayed on a preset display terminal.

6. The human-machine collaborative control method involving logistics storage and transportation as claimed in claim 1 is characterized in that: The method further comprises: Determine the operation items and the operation sequence of the operation items according to the preset production control items; Classifying the work tasks based on the work items and the work sequence to generate a work process type set; A job item queue is constructed according to the job flow type set.

7. The human-machine collaborative control method for logistics storage and transportation as claimed in claim 6, characterized in that: The step of constructing a job item queue according to the job flow type set includes: Extracting a plurality of operation processes and the operation items, operation sequence and initial conditions of each operation process according to the operation process type set; Constructing a work task according to the work items, work sequence and initial conditions of the work flow; wherein each of the work tasks includes a plurality of work items; A job item queue is constructed according to the job tasks.

8. The human-machine collaborative control method involving logistics storage and transportation as claimed in claim 1 is characterized in that: The method further comprises: Determine the operation items according to the preset production control items; The work items are classified according to their task execution attributes to generate a manual work item set and a feedback work item set.

9. A human-machine collaborative control system for logistics storage and transportation, characterized in that: The human-machine collaborative control system involving logistics storage and transportation includes: The operation task process control module is used to determine the target operation process according to the logistics storage and transportation task when receiving the logistics storage and transportation task; The operation task process control module is further used to determine the target operation project queue corresponding to the target operation process according to the operation project queue; The operation task process control module is also used to execute the logistics storage and transportation tasks according to the operation items and operation sequence of the target operation item queue; The job task process control module is also used to determine whether the target job item queue belongs to the manual job item set; The operation task process control module is further used to generate and issue a manual operation notification instruction when the target operation item queue belongs to the manual operation item set; The operation task process control module is also used to determine whether a confirmation signal based on the manual operation notification instruction feedback is received; A task status determination module, configured to update the status of the job item in the target job item queue to being executed upon receiving the confirmation signal; The job task process control module is also used to determine whether the target job item queue belongs to the feedback job item set; The operation task process control module is further used to determine whether a manual feedback completion signal is received when the target operation item queue belongs to the feedback operation item set; The task status determination module is further configured to update the status of the work item in the target work item queue to completed when receiving the manual feedback completion signal; The operation task process control module is also used to confirm that the logistics storage and transportation task is completed when the digital label of the operation project is the maximum digital label of the target operation project queue.

10. A human-machine collaborative control device for logistics storage and transportation, characterized in that: The human-machine collaborative control device involving logistics storage and transportation includes: a memory, a processor, and a human-machine collaborative control program involving logistics storage and transportation stored on the memory and executable on the processor. The human-machine collaborative control program involving logistics storage and transportation is configured to implement the human-machine collaborative control method involving logistics storage and transportation as described in any one of claims 1 to 8.

Images