Intelligent self-loading and unloading management method and device, electronic equipment and storage medium

Abstract

The invention discloses an intelligent self-loading and unloading management method and device, electronic equipment and a storage medium, which are used for solving the problem of low loading and unloading efficiency of the current self-loading and unloading related technology. Obtaining a to-be-loaded and unloaded cargo image, a cargo loading and unloading log and a monitoring video of a self-loading and unloading platform; performing cargo information matching according to the image of the cargo to be loaded and unloaded and the cargo loading and unloading log to obtain cargo attribute information of each cargo, and further combining the cargo loading and unloading log to generate a loading and unloading priority of each cargo; carrying out loading and unloading path identification on the monitoring video, and constructing a dynamic loading and unloading model; performing available loading and unloading path analysis according to the dynamic loading and unloading model, performing loading and unloading time sequence optimization decision based on each loading and unloading priority, and constructing a time sequence optimization path of cargo loading and unloading; and performing multi-task cooperative control calculation based on the time sequence optimization path to obtain a multi-task cooperative real-time control parameter and an operation efficiency value, performing dynamic loading and unloading management optimization according to the real-time control parameter and the operation efficiency value, and constructing an intelligent loading and unloading management model.

Description

Technical Field

[0001] The present invention relates to the technical field of loading and unloading management, and in particular to an intelligent self-loading and unloading management method, device, electronic equipment and storage medium. Background Art

[0002] With the rapid development of modern industrial automation technology, intelligent self-loading and unloading platforms are increasingly used in various production lines and logistics systems, becoming an important tool for improving production efficiency, reducing labor costs, and optimizing resource allocation. Self-loading and unloading platforms can complete the loading and unloading tasks of goods through automated systems, and realize efficient flow and collaborative operations between equipment and goods. Especially in the fields of high-speed logistics, warehouse management, and robot applications, they have broad application prospects.

[0003] However, with the complexity of the platform operating environment and the diversity of items, traditional static item loading and unloading management methods are often unable to cope with complex dynamic needs. These methods are usually based on fixed paths, preset tasks or simple rules. They cannot respond to changes in the platform environment, fluctuations in item status and external interference in real time. This leads to low loading and unloading efficiency, and even operational errors, item damage and other problems. Especially when multiple items need to be scheduled in real time, it is difficult to handle the dynamic relationship and scheduling conflicts between items using traditional methods, and cannot effectively guarantee the accuracy and timeliness of cargo loading and unloading.

[0004] To meet the above challenges, a more intelligent cargo loading and unloading management solution is urgently needed. Summary of the invention

[0005] The present invention provides an intelligent self-loading and unloading management method, device, electronic device and storage medium, which are used to solve or partially solve the problem that the current self-loading and unloading related technologies are unable to respond to platform environment changes, item status fluctuations and external interference in real time, resulting in low loading and unloading efficiency, and even operation errors, item damage and other problems.

[0006] The present invention provides an intelligent self-loading and unloading management method, the method comprising:

[0007] Obtain images of cargo to be loaded and unloaded, cargo loading and unloading logs, and monitoring videos of self-loading and unloading platforms;

[0008] Cargo information matching based on edge recognition is performed according to the cargo image to be loaded and unloaded and the cargo loading and unloading log to obtain cargo attribute information of each cargo;

[0009] Combine the cargo loading and unloading log and the attribute information of each cargo to perform dynamic loading and unloading priority calculation based on loading and unloading risk assessment, and generate the loading and unloading priority of each cargo;

[0010] Perform loading and unloading path recognition on the monitoring video and construct a dynamic loading and unloading model;

[0011] Analyze available loading and unloading paths according to the dynamic loading and unloading model, make loading and unloading timing optimization decisions based on each of the loading and unloading priorities, and construct a timing optimization path for cargo loading and unloading;

[0012] Based on the timing optimization path, multi-task collaborative control calculations are performed to obtain real-time control parameters and operational efficiency values ​​of multi-task collaboration, and dynamic loading and unloading management optimization is performed based on the real-time control parameters and the operational efficiency values ​​to construct an intelligent loading and unloading management model. The intelligent loading and unloading management model is used to make real-time decisions and management on cargo loading and unloading.

[0013] The present invention also provides an intelligent self-loading and unloading management device, comprising:

[0014] A data acquisition unit, used to acquire images of cargo to be loaded and unloaded, cargo loading and unloading logs, and monitoring videos of the loading and unloading platform;

[0015] A cargo information matching unit, configured to perform cargo information matching based on edge recognition according to the cargo image to be loaded and unloaded and the cargo loading and unloading log, to obtain cargo attribute information of each cargo;

[0016] A loading and unloading priority generation unit, configured to perform a dynamic loading and unloading priority calculation based on loading and unloading risk assessment in combination with the cargo loading and unloading log and each cargo attribute information, and generate a loading and unloading priority for each cargo;

[0017] A dynamic loading and unloading model building unit, used to identify the loading and unloading path of the monitoring video and build a dynamic loading and unloading model;

[0018] A loading and unloading path planning unit, used to analyze available loading and unloading paths according to the dynamic loading and unloading model, and make loading and unloading timing optimization decisions based on each of the loading and unloading priorities to construct a timing optimization path for cargo loading and unloading;

[0019] The intelligent loading and unloading management unit is used to perform multi-task collaborative control calculations based on the timing optimization path, obtain real-time control parameters and operational efficiency values ​​of multi-task collaboration, and perform dynamic loading and unloading management optimization based on the real-time control parameters and the operational efficiency values, and construct an intelligent loading and unloading management model. The intelligent loading and unloading management model is used to make real-time decisions and management on cargo loading and unloading.

[0020] The present invention also provides an electronic device, the device comprising a processor and a memory:

[0021] The memory is used to store program codes and transmit the program codes to the processor;

[0022] The processor is used to execute the intelligent self-loading and unloading management method as described in any one of the above items according to the instructions in the program code.

[0023] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program codes, and the program codes are used to execute the intelligent self-loading and unloading management method as described in any one of the above items.

[0024] It can be seen from the above technical solutions that the present invention has the following advantages:

[0025] An intelligent self-loading and unloading management method is provided. The first step is to obtain the image of the cargo to be loaded and unloaded, the cargo loading and unloading log and the monitoring video of the self-loading and unloading platform; the cargo information is matched based on edge recognition according to the image of the cargo to be loaded and unloaded and the cargo loading and unloading log, and the cargo attribute information of each cargo is obtained. Therefore, through the image edge segmentation technology, the shape and appearance characteristics of each cargo are accurately extracted to eliminate the risk of misidentification. And the accurate comparison with the loading and unloading log can ensure that the cargo attribute information obtained by the system is consistent with the actual cargo, thereby providing accurate basic data for subsequent operations. The second step is to combine the cargo loading and unloading log and the attribute information of each cargo to perform dynamic loading and unloading priority calculation based on loading and unloading risk assessment, and generate the loading and unloading priority of each cargo. Therefore, by comprehensively evaluating the physical properties of the cargo, the system can automatically identify high-risk cargo and ensure that special treatment measures are taken during loading and unloading to avoid damage and accidents. With the changes in the environment, platform load and operating conditions, the system can adjust the loading and unloading priority in real time. Some cargo needs to be handled first under high load conditions. The system will dynamically adjust the task priority based on real-time feedback to improve the overall operation efficiency and flexibility of the platform. Step 3: Identify the loading and unloading paths of the monitoring video and build a dynamic loading and unloading model. Based on the constructed dynamic twin model, the system continuously optimizes and adjusts the operation strategy according to the real-time feedback data to ensure the smooth progress of the entire loading and unloading process. In the subsequent processing flow, the system is able to analyze the optimal loading and unloading path for each cargo according to the loading and unloading priority of the cargo. Step 4: Analyze the available loading and unloading paths according to the dynamic loading and unloading model, make loading and unloading timing optimization decisions based on each loading and unloading priority, and build a timing optimization path for cargo loading and unloading. Therefore, through the dynamic optimization path planning of cargo loading and unloading, conflicts and intersections between paths can be avoided, ensuring that the robot arms and other equipment can perform tasks smoothly during subsequent loading and unloading. Step 5: Perform multi-task collaborative control calculations based on the timing optimization path to obtain real-time control parameters and operation efficiency values ​​for multi-task collaboration. Therefore, by comprehensively considering multiple factors such as cargo loading and unloading priority, path planning, load status, etc., the system can not only complete the task in the shortest time, significantly improve the overall efficiency and accuracy of loading and unloading operations, but also coordinate the collaborative work of multiple robot arms in different loading and unloading tasks to avoid conflicts, waiting and invalid operations between robot arms. Step 6: Perform dynamic loading and unloading management optimization based on real-time control parameters and operational efficiency values, and build an intelligent loading and unloading management model for real-time decision-making and management of cargo loading and unloading. Through dynamic optimization based on efficiency values, the system can adjust the operation strategy according to the feedback data of each operation and build an intelligent loading and unloading management model to facilitate the system to make better real-time decisions and management of cargo loading and unloading. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A flowchart of the steps of an intelligent self-loading and unloading management method;

[0028] Figure 2 It is a flow chart of cargo information matching steps based on edge recognition;

[0029] Figure 3 It is a flow chart of the steps of dynamic loading and unloading priority calculation based on loading and unloading risk assessment;

[0030] Figure 4 A flow chart of steps for identifying a loading and unloading path;

[0031] Figure 5 Construct a flowchart of the steps for a time-optimized path for cargo loading and unloading;

[0032] Figure 6 It is a flow chart of multi-task collaborative control calculation steps;

[0033] Figure 7 It is a flow chart of the optimization steps of dynamic loading and unloading management;

[0034] Figure 8 The figure is a structural block diagram of an intelligent self-loading and unloading management device. DETAILED DESCRIPTION

[0035] The embodiments of the present invention provide an intelligent self-loading and unloading management method, device, electronic device and storage medium, which are used to solve or partially solve the problem that the current self-loading and unloading related technologies are unable to respond to platform environment changes, item status fluctuations and external interference in real time, resulting in low loading and unloading efficiency, and even operation errors, item damage and other problems.

[0036] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] As an example, with the complexity of the platform operating environment and the diversity of items, traditional static item loading and unloading management methods are often unable to cope with complex dynamic needs. These methods are usually based on fixed paths, preset tasks or simple rules. They cannot respond to changes in the platform environment, fluctuations in item status, and external interference in real time. This leads to low loading and unloading efficiency, and even operational errors, item damage and other problems. Especially when multiple items need to be scheduled in real time, it is difficult to handle the dynamic relationship and scheduling conflicts between items using traditional methods, and cannot effectively guarantee the accuracy and timeliness of cargo loading and unloading.

[0038] Therefore, one of the core inventions of the embodiment of the present invention is to propose an intelligent cargo loading and unloading management method in view of the shortcomings of the current technology. (1) Through the image edge segmentation technology, the shape and appearance features of each cargo are accurately extracted to eliminate the risk of misidentification. The accurate comparison with the loading and unloading log can ensure that the cargo attribute information obtained by the system is consistent with the actual cargo, thereby providing accurate basic data for subsequent operations. (2) Through a comprehensive evaluation of the physical properties of the cargo, the system can automatically identify high-risk cargo and ensure that special handling measures are taken during loading and unloading to avoid damage and accidents. As the environment, platform load and operating conditions change, the system can adjust the loading and unloading priority in real time. Some cargo needs to be processed first under high load conditions. The system will dynamically adjust the task priority based on real-time feedback to improve the overall operation efficiency and flexibility of the platform. (3) Based on the constructed dynamic twin model, the system continuously optimizes and adjusts the operation strategy based on real-time feedback data to ensure that the entire loading and unloading process proceeds smoothly. In the subsequent processing flow, the system is able to analyze the optimal loading and unloading path for each cargo based on the cargo loading and unloading priority. (4) Through dynamic optimization path planning for cargo loading and unloading, conflicts and intersections between paths can be avoided, ensuring that the robotic arms and other equipment can perform tasks smoothly during subsequent loading and unloading. (5) By comprehensively considering multiple factors such as cargo loading and unloading priority, path planning, and load status, the system can not only complete the task in the shortest time, significantly improving the overall efficiency and accuracy of loading and unloading operations, but also coordinate the collaborative work of multiple robotic arms in different loading and unloading tasks, avoiding conflicts, waiting, and invalid operations between robotic arms. (6) Through dynamic optimization based on efficiency values, the system can adjust the operation strategy according to the feedback data of each operation and build an intelligent loading and unloading management model to facilitate the system to make better real-time decisions and management of cargo loading and unloading.

[0039] The intelligent self-loading and unloading management method proposed in the present invention is applicable to mechanical equipment equipped with a self-loading and unloading platform, a data processing platform, a cloud server node, and a network upload device. These can all be regarded as general computing nodes of the present invention. The data processing platform includes but is not limited to at least one of an audio and image management system, an information management system, and a cloud data management system.

[0040] Reference Figure 1 , shows a flowchart of the steps of an intelligent self-loading and unloading management method provided by an embodiment of the present invention, which may specifically include the following steps:

[0041] Step 101, obtaining images of cargo to be loaded and unloaded, cargo loading and unloading logs, and monitoring videos of the loading and unloading platform;

[0042] High-resolution cameras, such as DSLR (Digital Single Lens Reflex) or industrial cameras, can be used to capture images of the cargo to be loaded and unloaded. Set the resolution to at least 1920x1080 pixels so that the details of the captured images are clear. Choose a suitable lens to obtain clear images at different shooting distances and angles. Wide-angle lenses can be used to cover a larger field of view. Arrange cameras in the cargo loading and unloading area with the goal of fully covering the loading and unloading area. At the same time, in order to make the lighting uniform and avoid the influence of shadows and reflections on the image quality, you can set shooting parameters such as aperture, shutter speed and ISO (the sensitivity setting of the camera sensor to light).

[0043] Record cargo loading and unloading logs, including cargo ID, loading and unloading time, operator, loading and unloading method (such as manual or mechanical), etc. Use a spreadsheet system (such as Excel) or dedicated logistics management software to record. Recording can make the log information consistent with the timestamp of the image for subsequent matching and analysis.

[0044] For the acquisition of monitoring videos, you can choose a suitable high-definition industrial camera. For example, a camera with a resolution of 1920x1080. The camera supports real-time video recording at 30 frames per second to capture clear and smooth images. At the same time, the selected camera needs to have good low-light performance to meet the monitoring needs under different lighting conditions. Install cameras at key locations on the self-loading and unloading platform to cover the entire loading and unloading area. The camera's field of view should be able to capture all loading and unloading paths and platform components. Set the angle and height of the camera according to the actual site to avoid blind spots. To ensure the stability of the monitoring video, you can use anti-shake brackets and protective covers for protection. Real-time acquisition of monitoring video streams through video acquisition software (such as OpenCV or dedicated industrial camera software). And set the video encoding format (such as H.264) to optimize storage and transmission efficiency. Real-time monitoring video frame rate and resolution can stably obtain high-quality video streams under different conditions. To further ensure the security and accessibility of data, the real-time video stream can be saved to a local server or cloud storage, and backed up regularly to prevent data loss. Storage policies can also be set. For example, a video clip is saved every hour for subsequent analysis and processing.

[0045] Step 102, performing cargo information matching based on edge recognition according to the cargo image to be loaded and unloaded and the cargo loading and unloading log, to obtain cargo attribute information of each cargo;

[0046] In some embodiments, in combination Figure 2 The step flow chart shown in the figure shows that the implementation process of performing edge recognition-based cargo information matching according to the cargo images to be loaded and unloaded and the cargo loading and unloading logs to obtain cargo attribute information of each cargo can be implemented by executing the following sub-steps S01 to S05:

[0047] Step S01: segment the cargo edge contours of the cargo to be loaded and unloaded, and extract the edge contour lines of each cargo;

[0048] In this step, the OpenCV library is used to process the acquired image. First, the color image is converted to a grayscale image to simplify the subsequent edge detection process. For example, this can be achieved through cv2.cvtColor(image, cv2.COLOR_BGR2GRAY). Gaussian blur (such as a 5×5 Gaussian kernel) is applied to reduce the impact of image noise. This can be processed through cv2.GaussianBlur(gray_image, (5, 5), 0).

[0049] Use the Canny edge detection algorithm to extract the edges of the goods. Set the low threshold and high threshold (such as 50 and 150) to detect the edges in the image respectively. This can be achieved through cv2.Canny(blurred_image, 50, 150). Through edge detection, a binary image can be generated. The edge part is white and the other parts are black.

[0050] Step S02: Performing geometric analysis on each edge contour line to generate geometric features of each product;

[0051] Use cv2.findContours function to extract edge contours and return the contour list and hierarchy information. In the case of extracting only external contours, you can set the contour retrieval mode to cv2.RETR_EXTERNAL. Process each extracted contour and calculate its area and perimeter to filter out small noise contours. Specifically, for each edge contour of the goods, you can use cv2.contourArea to calculate its area and cv2.arcLength to calculate its perimeter.

[0052] The moment of the contour is obtained through the cv2.moments function, and the centroid position and shape features (such as rectangularity, roundness, etc.) are calculated. Then the extracted geometric features (such as area, perimeter, centroid coordinates, rectangularity, roundness, etc.) can be stored in a dictionary or data frame for subsequent analysis and matching.

[0053] Step S03: Perform cargo surface texture recognition on the cargo image to be loaded and unloaded, and extract cargo surface texture features;

[0054] To ensure data consistency, set the unit of each geometric feature (such as area in square centimeters). Use the Local Binary Pattern (LBP) or Gray-Level Co-occurrence Matrix (GLCM) method to extract texture features. Select appropriate parameters (such as neighborhood radius, direction) to extract effective texture information. For example, the cv2.calcHist function in OpenCV is used to calculate the histogram of the image to obtain texture distribution characteristics. For each cargo image, the gray-level co-occurrence matrix is ​​used to calculate texture features (such as contrast, correlation, energy, and uniformity). Exemplarily, the directions are set to 0, 45, 90, and 135 degrees to calculate texture features in multiple directions. The extracted cargo surface texture features are stored in the same data structure as the geometric features for subsequent processing.

[0055] Step S04: performing deep visual recognition based on each geometric morphological feature and surface texture feature of the goods, respectively, to generate a deep visual representation of each of the goods;

[0056] Select a deep learning model for visual recognition, such as the commonly used Convolutional Neural Network (CNN). For model training, set the input layer to the combination of the extracted geometric features and the surface texture features of the goods, and the output layer to the category label of the goods. Combine the geometric features of each product with the surface texture features of the goods into a feature vector, and together with the corresponding label, form a training data set. To ensure that the number of samples in each category is balanced, set training parameters such as batch size (32) and learning rate (0.001), and use the cross entropy loss function for training. During the training process, the performance of the model can be monitored through the validation set, and the accuracy and loss values ​​can be recorded. Use the confusion matrix to evaluate the performance of the model on different categories.

[0057] The geometric features and surface texture features of the goods extracted in the previous steps are input into the trained model for deep visual recognition, and the deep visual representation of each product can be output.

[0058] By processing cargo images through edge contour segmentation algorithms, the shape, size and edge features of each cargo can be accurately identified, ensuring the accuracy and reliability of image processing. In this way, the physical characteristics of cargo are extracted in real time, providing accurate data for subsequent task scheduling and path planning.

[0059] Step S05: Cargo information is matched with the cargo loading and unloading log according to each deep visual representation to obtain cargo attribute information of each cargo.

[0060] Cargo information is matched by deep visual representation (e.g., feature vector) with the information in the cargo loading and unloading log. For example, the similarity between feature vectors is evaluated by using metrics such as cosine similarity or Euclidean distance. A matching threshold (e.g., 0.8) is set, and a match is determined to be successful when the similarity between the two is higher than the matching threshold. The attribute information of the successfully matched cargo (e.g., cargo ID, type, weight, size, etc.) is recorded in the database for subsequent query and analysis. In addition, a matching report can be generated to record the correspondence between the deep visual representation of each cargo and the information in the loading and unloading log.

[0061] Thus, by matching cargo loading and unloading logs and combining the attribute information of each cargo (such as weight, fragility, etc.), the system can fully understand the characteristics of the items. This provides basic data support for subsequent risk assessment, priority sorting, and loading and unloading path planning, avoiding improper loading and unloading or operational accidents caused by misidentification or missing data.

[0062] Step 103, performing dynamic loading and unloading priority calculation based on loading and unloading risk assessment in combination with the cargo loading and unloading log and the attribute information of each cargo, and generating the loading and unloading priority of each cargo;

[0063] In some embodiments, in combination Figure 3 The step flow chart shown in the figure combines the cargo loading and unloading log and the attribute information of each cargo to perform dynamic loading and unloading priority calculation based on loading and unloading risk assessment, and generates the implementation process of the loading and unloading priority of each cargo, which can be achieved by executing the following sub-steps S11 to S15:

[0064] Step S11: Perform loading and unloading risk assessment on each cargo attribute information respectively to generate a loading and unloading risk assessment value for each cargo;

[0065] Identify the key factors that affect the risk during loading and unloading. These include cargo type, weight, volume, loading and unloading method (such as manual or mechanical), environmental conditions (such as weather, temperature), etc. Classify these factors and set the weight of each factor. For example, weight and volume have a greater impact on risk, so they are given higher weights (such as 0.4 and 0.3). Environmental conditions are given lower weights (such as 0.2). Set risk assessment indicators based on the identified risk factors.

[0066] Extract the required data from the cargo attribute information, including cargo type, weight, volume, etc., and format it into calculable values. Use data processing tools to clean and convert the data to ensure data accuracy and consistency. Apply the set risk assessment model to each cargo, calculate the loading and unloading risk assessment value of each cargo, and store the results in the database for subsequent query and analysis.

[0067] Step S12: Cargo flow mining is performed based on cargo loading and unloading logs to generate flow data for each cargo;

[0068] Extract relevant data (such as cargo ID, cargo type, loading and unloading time, loading and unloading method, etc.) from cargo loading and unloading logs. Ensure data accuracy and completeness through data cleaning and conversion (such as deduplication and formatting). Set a time window (such as 24 hours) to analyze the flow of cargo within this time period. Use techniques such as frequency analysis and path analysis to extract the pattern of cargo flow. Count the frequency of specific cargo arriving from one location to another within a specific time period. Store the generated flow data in the database for subsequent query and analysis.

[0069] Step S13: Calculate the final placement position according to each flow direction data to generate the final placement position of each cargo;

[0070] Identify factors that affect the final placement of goods, including flow data, warehouse layout, cargo type, loading and unloading sequence, etc. Set the weight of each factor so that the influence of different factors can be considered in the calculation. Choose a suitable algorithm for position calculation. For example, rule-based algorithms and optimization algorithms (such as genetic algorithms or particle swarm optimization) can be used.

[0071] Set the objective function, such as minimizing loading and unloading time and path length. Input the flow data and other relevant factors into the location calculation model, and ensure that all data formats are consistent for calculation. For example, use the SciPy library in Python or other optimization tools to calculate the placement. After generating the final placement of each cargo, the results can be stored in a database for subsequent use.

[0072] Step S14: Perform real-time platform loading and unloading load analysis according to the cargo loading and unloading log to obtain real-time loading and unloading load data;

[0073] Set up visualization tools to display the placement of goods to help operators understand and adjust. Define the calculation criteria for loading and unloading loads, including the number, weight, and volume of goods loaded and unloaded each time. According to the information in the cargo loading and unloading log, set the time window for real-time load analysis (such as every hour as the time window). Extract relevant data for each loading and unloading from the cargo loading and unloading log (such as cargo ID, cargo type, cargo weight, cargo quantity, etc.), and improve data accuracy through data cleaning and filtering. Use data processing tools to calculate the total load in each time window. Generate real-time loading and unloading load data based on the calculation results, including information such as current load, maximum load, and remaining load.

[0074] Step S15: dynamically calculate the loading and unloading priority of each cargo based on the loading and unloading risk assessment value and the final placement position of each cargo according to the real-time loading and unloading load data, and generate the loading and unloading priority of each cargo.

[0075] An alarm mechanism can be set. When the real-time load approaches or exceeds the maximum load, the relevant personnel are notified immediately. First, determine the factors that affect the loading and unloading priority, including the loading and unloading risk assessment value, cargo flow direction, real-time loading and unloading load data, and cargo placement location. Set the weight of each factor to ensure that the priority calculation can reflect the actual situation. Generate the loading and unloading priority of each cargo according to the weighted method of the influencing factors and sort them by priority. If the weight values ​​of the influencing factors of the loading and unloading risk assessment value, cargo flow direction, real-time loading and unloading load data, and cargo placement location of a certain cargo are w1, w2, w3, and w4 respectively, then the calculation result of its loading and unloading priority is w1+w2+w3+w4. The larger the value, the higher the loading and unloading priority. The calculated results are stored in the database to facilitate subsequent loading and unloading operations. In the actual loading and unloading process, the calculated loading and unloading priority information can be used to guide operators to load and unload, so that high-risk cargo and key cargo can be handled first.

[0076] Therefore, by dynamically analyzing the attribute information of each cargo (such as fragile, heavy, large volume, etc.), the loading and unloading risk of each cargo can be intelligently assessed. High-risk items (such as fragile items, heavy cargo, etc.) will be marked as high priority to ensure that these items can receive priority attention and processing during the loading and unloading process to avoid damage or accidents. Combined with the real-time load and platform operating conditions, the system can dynamically calculate and adjust the loading and unloading priority of the cargo. In the embodiment of the present invention, the loading and unloading priority of the items not only depends on the preset static rules, but also is intelligently adjusted according to the real-time data to ensure the optimal scheduling of resources and efficient operation.

[0077] Step 104, identifying the loading and unloading path of the monitoring video and constructing a dynamic loading and unloading model;

[0078] In some embodiments, in combination Figure 4The step flow chart shown in the figure, the implementation process of identifying the loading and unloading path of the monitoring video and building a dynamic loading and unloading model can be achieved by executing the following sub-steps S21 to S25:

[0079] Step S21: performing visual recognition of loading and unloading paths on the monitoring video to obtain multiple loading and unloading paths;

[0080] First, the acquired monitoring video is preprocessed, including noise removal, image enhancement and grayscale conversion, to improve the accuracy of subsequent analysis. Through Gaussian blur and histogram equalization technology in OpenCV, the video is processed in frames, each frame is extracted and analyzed, and the processing time of each frame is set (such as 100ms) to ensure real-time performance.

[0081] Based on each extracted image frame, the loading and unloading paths are identified through computer vision algorithms (such as Hough transform or Canny edge detection). Specifically, multiple loading and unloading paths are extracted by detecting straight lines and edges in the image. Relevant parameters (such as line length and angle threshold) are set to ensure that the detected paths meet the requirements of actual loading and unloading operations. The identified multiple loading and unloading paths are marked and stored as a path data structure, which contains information such as the starting point, end point and path length of the path. Subsequently, a path visualization diagram can be generated based on the loading and unloading path to help operators understand and confirm the recognition results.

[0082] By monitoring video data, the platform can obtain real-time changes in the working environment, including obstacle locations, working space, and path planning. Based on visual recognition technology, the system accurately identifies and analyzes loading and unloading paths to ensure the correctness of path selection and avoid misoperation due to visual obstructions or environmental changes.

[0083] Step S22: performing platform component analysis on the monitoring video to obtain multiple platform components;

[0084] Use deep learning models such as YOLO (You Only Look Once) or SSD (Single Shot MultiBox Detector) to identify and extract platform components. Specifically, select pre-trained models to improve recognition efficiency and fine-tune them to suit specific application scenarios. Set a detection threshold (for example, 0.5) to ensure that the identified platform components have a certain degree of confidence. For each identified platform component, extract its feature information, including component type, size, location, and status. Extract the edge information of the component through image processing (such as contour detection) and store it in a structured database for subsequent query and analysis.

[0085] Step S23: performing path obstacle monitoring according to multiple platform components and marking each path obstacle node;

[0086] Determine the definition of path obstacles, including any objects that block the loading and unloading path. Such as unmarked cargo, tools, individual platform components or other equipment, etc. Use image segmentation (such as GrabCut or deep learning segmentation models) to identify path obstacles in the video frame. Mark each identified path obstacle as a node and record its location and size information in the video. Store the marked obstacle information in the database, including the coordinates, type and related loading and unloading path of the obstacle.

[0087] Step S24: performing spatial location positioning on each of the path obstacle nodes to generate spatial location coordinates;

[0088] Computer vision technology (such as stereo vision or depth camera) is used to locate the spatial position of obstacles. At the same time, camera calibration is combined to ensure the accuracy of position detection. Specifically, the camera internal and external parameters are set to optimize the spatial positioning process. For each path obstacle node, its position coordinates (X, Y, Z) in three-dimensional space are calculated. The obstacle position is predicted by triangulation or deep learning model. The calculated spatial position coordinates are stored in the database for subsequent analysis and calculation.

[0089] Step S25: Perform three-dimensional topological modeling based on multiple loading and unloading paths and each spatial position coordinate to construct a dynamic loading and unloading model.

[0090] Select a suitable 3D modeling tool to construct a twin model of the dynamic loading and unloading platform (referred to as the dynamic loading and unloading model in the present invention). The 3D modeling tool used must support the import of coordinate data and path information. A 3D model is generated by combining the spatial position coordinates of the path obstacle nodes and multiple loading and unloading path information through a point cloud-based modeling method.

[0091] Import the spatial coordinates of the loading and unloading path information and the path obstacle nodes into the 3D modeling tool to generate the 3D topology of the dynamic loading and unloading platform. Optimize the generated 3D model to ensure that it can reflect the dynamic changes of the platform (such as cargo loading and unloading status, path changes, etc.) in real time. At the same time, through model verification methods (such as comparing the spatial data measured on the spot with the spatial data of the model), ensure that the model accurately reflects the layout and status of the actual loading and unloading platform. Applying the dynamic loading and unloading model to the intelligent loading and unloading management system can provide support for loading and unloading decision-making and management.

[0092] Specifically, the implementation process of constructing a dynamic loading and unloading model by performing three-dimensional topological modeling based on multiple loading and unloading paths and each spatial position coordinate can be achieved by executing the following sub-steps S251 to S259:

[0093] Step S251: Identify the current cargo placement location;

[0094] High-definition cameras and LiDAR (Light Detection and Ranging) sensors are used to collect images and depth data of the cargo placement in real time. To ensure that real-time cargo information is captured, the camera resolution can be set to 1920x1080 and the LiDAR sampling frequency can be set to 10 Hz. The camera is calibrated to ensure that it is consistent with the LiDAR coordinate system for subsequent data fusion.

[0095] The edges of the goods are identified through image processing algorithms (such as edge detection and contour extraction), and three-dimensional positioning is performed in combination with depth information. Specifically, the Canny edge detection and cv2.findContours function in OpenCV can be used to extract the contours of the goods. The image data and point cloud data are combined to generate the three-dimensional coordinates of the goods, and the placement position of each item is recorded and stored in the database.

[0096] Step S252: Calculate the platform operation space based on the current cargo placement position and each spatial position coordinate to obtain the platform operation space characteristics;

[0097] Define the workspace as the area that the loading and unloading robot can reach when performing tasks. Consider the location of the cargo and the location of obstacles to construct the boundaries of the workspace. Set the calculation of the workspace based on the three-dimensional coordinates of the cargo (i.e. the current cargo location) and the coordinates of the obstacles (i.e. the spatial position coordinates of the path obstacle nodes), and determine the effective area of ​​the space through geometric calculation methods. Use a space partitioning algorithm (such as a Voronoi diagram) to calculate the workspace.

[0098] First, a spatial model is established based on the coordinates of the cargo and obstacles to calculate the accessible area. Calculation parameters, such as the minimum separation distance, are set to ensure that the robot arm maintains a safe distance from obstacles during movement. Then, the platform operating space features are extracted, including the volume, boundary shape, and accessibility of the effective operating area. These extracted features are stored in a database for subsequent analysis and application.

[0099] Step S253: performing platform component spatial layout analysis on multiple platform components to generate spatial layout data of the platform components;

[0100] Determine the spatial layout characteristics of platform components, including component type, size, relative position, and layout. Platform components include robotic arms, sensors, cargo storage areas, etc. Set the reference coordinate system for layout analysis to facilitate subsequent spatial calculations. Process the spatial layout data of platform components through structured data analysis tools. Store the size and position of each platform component as a data frame for subsequent analysis. Use rule-based analysis methods to check the rationality of platform component layout, including spacing and arrangement. Generate spatial layout data for each platform component, including the platform component's starting position, bounding box, and occupied space. Store this data in a database for subsequent query and analysis.

[0101] Step S254: performing a three-dimensional topological structure analysis of the platform based on the spatial layout data to obtain three-dimensional topological structure features;

[0102] The three-dimensional topology is defined as the spatial relationship between platform components, including relative position, distance and connection mode. The analysis of the three-dimensional topology is based on the spatial layout data of the platform components. The basic concepts of graph theory can be used to regard the platform components as nodes, and the lines between the nodes represent the mutual relationship. Graph algorithms (such as Dijkstra or A* algorithms) are used to analyze the connection relationship between platform components and calculate the optimal path and relative position relationship.

[0103] Set the minimum connection distance to ensure the reasonable connection between platform components. Extract the three-dimensional topological structure features, including the number of nodes, number of edges, connectivity and network density. Store the three-dimensional topological structure features in the database for subsequent analysis and query. In addition, the topological structure can be visualized through visualization tools to help operators understand the structural relationship.

[0104] Step S255: obtaining state parameters of the loading and unloading robot arm of the loading and unloading platform;

[0105] Install a variety of sensors on the robot arm, including position sensors, force sensors, and angle sensors, to monitor the state parameters of the robot arm in real time. Specifically, set the sensor sampling frequency (e.g., 10 times per second) to ensure the accuracy and timeliness of the state data. Obtain sensor data through a data acquisition system (such as a PLC (Programmable Logic Controller) or a single-chip microcomputer) to record the state parameters of the robot arm in real time, including position, speed, acceleration, and load. Store the collected data in a database for subsequent analysis.

[0106] Step S256: performing operation status analysis on the state parameters of the loading and unloading robot arm to obtain the operation status characteristics of the robot arm;

[0107] Analyze the operating status of the robot arm and identify its working efficiency, load conditions, and motion trajectory. Set analysis indicators, including working cycle, average load, and motion smoothness. Extract key features from the state parameters as the robot arm operating status characteristics, such as average speed, maximum load, operating time, and cycle frequency. Calculate these features through statistical analysis methods. At the same time, graphical tools can be used to visualize these data to help identify the changing trends of state characteristics.

[0108] Step S257: Calculate the loading and unloading range of the robot arm according to the operating state characteristics of the robot arm, and extract the loading and unloading range of the robot arm;

[0109] According to the extracted robot arm operation status features, the robot arm operation status is classified through machine learning models (such as support vector machine (SVM) or random forest) to identify normal, overloaded and faulty states. The accuracy threshold of the model is set (such as 85%) to ensure the reliability of the classification results.

[0110] Determine the range of motion of the robot arm for loading and unloading, including the spatial area where the robot arm can operate effectively. Take into account the working radius and swing angle of the robot arm. Determine the geometric shape of the range of motion based on the model and motion parameters of the robot arm (such as joint angle, telescopic length, etc.), usually a cone or a sphere. To ensure the accuracy of the calculation results, the kinematic model of the robot arm (such as the Denavit-Hartenberg parameter method) can be used to calculate the range of motion. Set the calculation accuracy (such as 0.01 meters) to ensure the level of detail of the range of motion. Use a 3D visualization tool to display the range of motion of the robot arm to help operators understand and adjust the working area of ​​the robot arm.

[0111] Step S258: performing 3D point cloud modeling on the 3D topological structure features according to the platform operation space features to construct a 3D loading and unloading platform model;

[0112] Generate a three-dimensional model of the activity range to facilitate subsequent dynamic rendering and analysis. Specifically, according to the platform's operating environment (such as obstacle location, spatial layout) and the range of motion of the robotic arm, extract the characteristics of the operating space and identify the operable area and the prohibited area. Set a feature extraction algorithm (such as the Random Sample Consensus (RANSAC)) to analyze the point cloud data in the operating space. To ensure the density of the point cloud data (such as at least 10 points per square meter) to ensure the accuracy of the model, a lidar or depth camera can be used to obtain the point cloud data of the operating space. Then filter and denoise the point cloud data, and use the filtering algorithm in the Point Cloud Library (PCL) to remove abnormal points. Construct a three-dimensional loading and unloading platform model based on the processed point cloud data. At the same time, apply the Delaunay triangulation or Poisson reconstruction algorithm to generate a surface model.

[0113] Step S259: According to the loading and unloading range of the robot arm and multiple loading and unloading paths, the three-dimensional loading and unloading platform model is dynamically rendered to construct a dynamic loading and unloading model.

[0114] In this step, the authenticity and operability of the three-dimensional loading and unloading platform model are ensured by refining and optimizing the model. Specifically, a suitable dynamic rendering tool can be selected to simulate and render the loading and unloading behavior. The selected dynamic rendering tool must support physical simulation and real-time rendering. Use a physical engine to perform dynamic simulation of the model to ensure the realism of the loading and unloading behavior. According to the loading and unloading range of the robot arm and multiple loading and unloading paths, the loading and unloading behavior of the robot arm is modeled, including actions such as grasping, moving and releasing. Set action parameters (such as speed, acceleration, etc.) to ensure the smoothness and consistency of the simulation results. Integrate the dynamic behavior of the robot arm with the three-dimensional loading and unloading platform model, and display the loading and unloading process through real-time simulation. At the same time, record key data (such as time, energy consumption, etc.) during the loading and unloading process to provide a basis for subsequent optimization.

[0115] By building a dynamic twin model of the self-loading and unloading platform, the virtual model and the actual operating platform can be synchronized in real time to reflect the actual operating status of the platform. The twin model provides high-precision, real-time updated virtual environment data for subsequent path planning, task scheduling, equipment coordination, etc., further improving the decision-making accuracy and response speed of the system.

[0116] Step 105, analyzing available loading and unloading paths according to the dynamic loading and unloading model, and making loading and unloading timing optimization decisions based on each of the loading and unloading priorities, to construct a timing optimization path for cargo loading and unloading;

[0117] In some embodiments, in combination Figure 5The step flow chart shown in the figure analyzes available loading and unloading paths according to the dynamic loading and unloading model, makes loading and unloading timing optimization decisions based on various loading and unloading priorities, and constructs an implementation process of the timing optimization path for cargo loading and unloading, which can be achieved by executing the following sub-steps S31 to S35:

[0118] Step S31: Analyze the available loading and unloading paths for each cargo one by one according to the dynamic loading and unloading model, and generate the available loading and unloading paths for each cargo;

[0119] According to the dynamic loading and unloading model constructed above, the real situation of the loading and unloading environment can be accurately reflected, including cargo, robotic arms, and path information. First, determine the algorithm for path analysis, such as the A* algorithm or the Dijkstra algorithm. These algorithms can effectively find the shortest path and take obstacles into account. For each cargo, the selected path algorithm is used to analyze the available loading and unloading paths from the current placement position to the target placement position.

[0120] Set the cost function of the path, including factors such as time, distance, and obstacle impact. In the path calculation, you can set a threshold (such as the maximum allowed path length) to exclude unreasonable paths. Mark the available loading and unloading paths for each cargo searched and store them in the database, including information such as the starting point, end point, nodes passed, and path cost of the path. At the same time, a visual diagram can be generated to help operators understand the available loading and unloading paths for each cargo.

[0121] Step S32: Cargo transportation intersection prediction is performed for each available loading and unloading path, and the loading and unloading path intersection points are marked;

[0122] Determine the definition of intersections, including nodes where the loading and unloading paths of different goods intersect. The identification of intersections is crucial to optimizing the loading and unloading process. Intersection detection algorithms in graph theory, such as graph-based edge intersection detection methods, can be used to detect intersections for each available loading and unloading path for each cargo, record the intersection nodes between the paths, and calculate the impact of the intersection (such as traffic flow). Set marking rules for intersections. For example, when an intersection has multiple paths and the flow exceeds a set threshold, mark it. Store the marked intersection information in a database, including information such as the coordinates of the intersection, the paths involved, and the intersection flow. At the same time, a visual diagram can be generated to identify all intersections to help operators consider potential intersection conflicts when making decisions.

[0123] Step S33: Perform potential path obstacle detection on each available loading and unloading path to identify potential path obstacles;

[0124] Use computer vision technology (such as deep learning models) to detect obstacles on available loading and unloading paths. For example, use object detection algorithms such as YOLO or SSD to identify potential obstacles in available loading and unloading paths. To ensure that the identified obstacles are sufficiently reliable, you can set the detection accuracy (such as 95%). Perform real-time video analysis on the available loading and unloading paths for each cargo to detect obstacles on the path. Set a time window (such as 10 frames per second) to improve real-time performance. Mark the detected obstacles, including information such as the type, size, and location of the obstacle. Store the identified potential path obstacle information in a database for subsequent analysis and processing.

[0125] Step S34: Perform available path avoidance analysis based on potential path obstacles and loading and unloading path intersections to generate an avoidance path for each cargo;

[0126] Set avoidance strategies, including dynamic path adjustment and static obstacle avoidance. An improved version of the A* algorithm can be used, taking into account the impact of obstacles and intersections on the path. Specifically, set the parameters for avoidance path calculation, such as minimum safe distance and maximum turning angle, to ensure the safety and feasibility of the avoidance path. After detecting potential obstacles and intersections, use the path replanning algorithm to generate a new avoidance path. Ensure that the new path does not conflict with obstacles and minimize the path cost. For each cargo, generate an avoidance path based on its specific available loading and unloading path and mark it. Store the generated avoidance path information in the database, including information such as the starting point, end point, nodes passed through, and avoidance cost of the path.

[0127] Step S35: Optimizing the loading and unloading sequence for each avoidance path according to each loading and unloading priority, and constructing a cargo loading and unloading sequence optimization path.

[0128] Generate a visualization diagram to identify the avoidance path. According to the loading and unloading priority of each cargo, optimize the avoidance path in time. The timing optimization strategy is: consider the loading and unloading priority and avoidance path of the cargo at the same time (such as minimizing the avoidance cost and minimizing the number of nodes passed), and ensure that high-priority cargo is loaded and unloaded first under the lowest risk. Generate the loading and unloading timing optimization path through the scheduling algorithm (such as shortest job first or priority scheduling) to ensure the efficiency and safety of the operation.

[0129] The order and timing optimization of cargo loading and unloading is crucial. In this step, the loading and unloading paths of each cargo are optimized and analyzed based on the loading and unloading priorities of the cargo, and multiple feasible path solutions are proposed. By evaluating the intersections, obstacles and resource load conditions between different paths, the system accurately selects the optimal loading and unloading path, thereby improving loading and unloading efficiency and avoiding resource waste.

[0130] Step 106, performing multi-task collaborative control calculation based on the timing optimization path, obtaining real-time control parameters and operation efficiency values ​​of multi-task collaboration, and performing dynamic loading and unloading management optimization according to the real-time control parameters and the operation efficiency values, and constructing an intelligent loading and unloading management model, which is used to make real-time decisions and management on cargo loading and unloading.

[0131] In some embodiments, in combination Figure 6 The step flow chart shown in the figure performs multi-task collaborative control calculation based on the timing optimization path to obtain the implementation process of real-time control parameters and operation efficiency values ​​of multi-task collaboration, which can be achieved by executing the following sub-steps S41 to S45:

[0132] Step S41: performing collaborative operation control of the platform robot arm according to the timing optimization path, and collecting real-time control parameters of multi-task collaboration;

[0133] In the robot control system, a multi-task collaborative control architecture can be designed first. Specifically, to ensure that multiple robots can work together according to the loading and unloading timing optimization path, real-time control algorithms (such as PID control (Proportional-Integral-Derivative Control) or fuzzy control) can be used for precise control. Set control parameters, such as control frequency (for example, 50Hz), to achieve real-time response. Build a task scheduling mechanism so that each robot can assign tasks based on priority and available paths. Use priority-based scheduling algorithms (such as EDF scheduling (Earliest Deadline First)) to schedule loading and unloading tasks. Set scheduling parameters (such as task response time and maximum waiting time) to improve system efficiency.

[0134] During the collaborative control process, collect the real-time control parameters of the robot arm, including position, speed, acceleration, load, temperature, etc. and record the data through the data acquisition system. Configure the data acquisition frequency to ensure real-time performance (such as 100 acquisitions per second). Store the collected real-time control parameters in a database using a structured format (such as JSON or CSV) for subsequent analysis. In addition, a data backup strategy can be set to ensure the security and integrity of real-time data.

[0135] Step S42: Calculate the loading and unloading execution time for the real-time control parameters to generate the loading and unloading execution time for each cargo;

[0136] Define the loading and unloading execution time, including the total time from the start of loading and unloading to the completion of the robot. In the calculation process of loading and unloading execution time, factors such as action time, waiting time and delay need to be considered. Extract relevant data from real-time control parameters to calculate the loading and unloading execution time for each cargo. Set thresholds (such as the maximum allowed execution time) to ensure the validity of the data. At the same time, store the calculated loading and unloading execution time in the database, including the ID, execution time and related parameters of each cargo.

[0137] Step S43: acquiring multi-task collaborative operation simulation data, and performing loading and unloading conflict statistics on the operation simulation data to identify loading and unloading conflict time points;

[0138] Define loading and unloading conflicts, including multiple robotic arms trying to access the same loading and unloading path or operate the same cargo at the same time. Monitor the status of loading and unloading tasks and identify potential conflicts through event-driven models. Count the time points when conflicts occur through set operation simulation data, and set conflict identification algorithms (such as time series analysis) to monitor task status changes. Set a time window for conflict detection (such as once per second) to ensure that conflict events are captured in a timely manner. Record the identified loading and unloading conflict time points in the database, including the conflict time, the cargo involved, and the robotic arm information. Generate a visualization diagram to show the time series of conflicts to assist operators in analyzing the causes of conflicts.

[0139] Step S44: mining the conflict path distribution at the loading and unloading conflict time points to obtain the loading and unloading conflict path distribution data;

[0140] Determine the definition of the conflict path, including the path between the conflict time points, the robot arms involved in the conflict, and the cargo information. Perform path distribution analysis through data mining algorithms (such as cluster analysis or association rule algorithms). Extract relevant information from the loading and unloading conflict time point data and analyze the distribution of conflict paths. Set mining algorithm parameters (such as minimum support and confidence). Count the frequency of occurrence of each conflict path and identify high-frequency conflict paths. Store the mining distribution data of the loading and unloading conflict paths in the database, including path information, frequency, and information about related cargo and robot arms. Generate a visualization diagram to show the distribution of conflict paths and help optimize path planning.

[0141] Step S45: Quantitative calculation of the collaborative efficiency is performed based on the loading and unloading conflict path distribution data and each loading and unloading execution time to generate an operational efficiency value of multi-task collaboration.

[0142] Define the efficiency of multi-task collaborative operation, including factors such as loading and unloading execution time, conflict path distribution and task completion rate. Specifically, it can be set as: E(collab)=T(exec)+C(path) / T(total). Among them, E(collab) is the collaborative efficiency (the operational efficiency value of multi-task collaboration), T(total) is the total task time, C(path) is the impact coefficient of the conflict path, and T(exec) is the loading and unloading execution time. Using data analysis tools, calculate the multi-task collaborative operation efficiency value of each cargo by integrating the loading and unloading execution time and loading and unloading conflict path distribution data. Set efficiency evaluation standards (such as efficiency values ​​greater than 0.8 are efficient) to ensure the reliability of the evaluation results. The calculated multi-task collaborative operation efficiency values ​​are stored in the database, including the ID, efficiency value and related parameters of each cargo.

[0143] By calculating the loading and unloading sequence of each cargo and dynamically adjusting it according to priority, path conflict, robot arm workload, etc., the system can generate an efficient and safe loading and unloading sequence for each cargo, thereby improving the overall efficiency of the operation and shortening the operation cycle. When multiple tasks are executed simultaneously, the system can coordinate the working order and load distribution of each robot arm to avoid conflicts or excessive waiting between robots. Through multi-task collaborative control, the movements of each robot arm can be synchronized with the movements of other robot arms to ensure maximum operation efficiency. Through real-time monitoring and quantitative calculation, the system can evaluate and generate a multi-task collaborative operation efficiency value based on indicators such as the work efficiency of the robot arm and the degree of task completion. This efficiency value provides a basis for subsequent optimization and adjustment, helping the platform to continuously improve the effect of collaborative operations.

[0144] In some embodiments, in combination Figure 7 The step flow chart shown in the figure performs dynamic loading and unloading management optimization according to real-time control parameters and operation efficiency values, and constructs an implementation process of an intelligent loading and unloading management model, which can be achieved by executing the following sub-steps S51 to S54:

[0145] Step S51: Predict loading and unloading failures according to real-time control parameters to generate loading and unloading failure prediction data;

[0146] In practical applications, a prediction model can be pre-trained for loading and unloading fault prediction.

[0147] First, multiple historical control parameters of the robot arm in the multi-task collaboration process are collected. To ensure the stability of model training, the collected data need to be preprocessed. This includes denoising, standardization, and missing value processing to obtain a data set for model training. Select a suitable fault prediction model, such as a classification model based on machine learning (such as random forest, support vector machine) or a deep learning model (such as long short-term memory network (LSTM)) for time series prediction. For the selected model, hyperparameters (such as the number of decision trees in the random forest, the depth of the long short-term memory network, etc.) can be set to optimize the model performance. Divide the prepared data set into a training set and a test set (such as 70% training and 30% testing) according to actual needs. Use the training set to train the model, and use the test set to evaluate the prediction accuracy of the trained model (the prediction target is more than 95%). Cross-validation (such as k-fold cross-validation) can also be used to further improve the robustness and accuracy of the model to ensure that the model can effectively identify potential faults.

[0148] Based on the trained loading and unloading fault prediction model, the real-time control parameters are used to predict loading and unloading faults and generate loading and unloading fault prediction data. The output prediction results include the predicted fault type, probability of occurrence and recommended preventive measures. The prediction results are stored in the database for subsequent analysis and fault tracing and location.

[0149] Step S52: tracing the fault source of the loading and unloading fault prediction data to obtain the fault tracing point of the self-loading and unloading platform;

[0150] Define the standards and processes for fault tracing, which can include identifying the cause of the fault from the prediction data. Fault tracing can be achieved through fault tree analysis (FTA) or causal relationship model. Set key parameters for tracing, such as the time window of fault occurrence and the scope of impact, to ensure the effectiveness of the tracing process.

[0151] According to the loading and unloading fault prediction data, the real-time parameters related to the fault are analyzed to identify the time point when the fault occurs and the equipment affected. Then, based on the pre-set risk threshold (such as the probability of failure greater than 0.7), high-risk parameters and equipment can be screened out. The determined fault tracing points are recorded in the database, including the traceable parameters, the time of occurrence, the related equipment and the predicted fault type. In addition, a traceability report can be generated to help operators understand the root cause of the fault and provide a basis for subsequent fault diagnosis.

[0152] Step S53: Performing instant fault diagnosis on the fault source tracing point to obtain an instant fault diagnosis result;

[0153] Select appropriate fault diagnosis methods based on actual needs. Common methods include rule-based expert systems, model-based diagnosis, and data-driven methods. Set standards and processes for fault diagnosis to ensure the accuracy and effectiveness of immediate fault diagnosis. Perform detailed analysis of real-time parameters at the fault source point. Specifically, identify abnormal behavior through time series analysis or anomaly detection algorithms (such as isolation forests). For better judgment, a fault diagnosis standard can be set in advance. For example, when the load exceeds the rated value or the temperature of the robot arm is abnormal, it is marked as a fault. Generate immediate fault diagnosis results based on the analysis results, including the fault type, impact level, and recommended maintenance measures.

[0154] By real-time monitoring of the platform's operating status and the robot's movements, the system can predict potential failure risks and issue alerts in advance. Through failure prediction, the platform can make adjustments before problems occur, reducing the occurrence of sudden failures and avoiding affecting the overall operation progress.

[0155] Step S54: Perform iterative optimization of dynamic loading and unloading management based on the operational efficiency value and the instant fault diagnosis results, and build an intelligent loading and unloading management model.

[0156] Design an intelligent loading and unloading management model, combine the operational efficiency value of multi-task collaboration and the results of instant fault diagnosis, and set optimization goals (such as minimizing loading and unloading time, reducing failure rate, etc.). Specifically, a system dynamics model or optimization algorithm (such as genetic algorithm, particle swarm optimization) can be used to design dynamic management strategies. The intelligent loading and unloading management model is verified by collecting historical data and real-time parameters, and the management strategy is optimized by adjusting the model parameters. Furthermore, a feedback mechanism can be set to ensure that the model can be continuously optimized according to new data. For example, during the optimization process, multiple iterations (such as 10 times) are performed, and the model performance is evaluated after each iteration to ensure continuous improvement of loading and unloading efficiency and failure rate. The optimized intelligent loading and unloading management model and its parameters are stored in the database. At the same time, a model performance evaluation report is generated, including data on improved loading and unloading efficiency and reduced failure rate.

[0157] Through dynamic optimization based on efficiency values, the system can adjust the operation strategy according to the feedback data of each operation and build an intelligent loading and unloading management model. The model continuously learns and adjusts, so that the platform can maintain efficient and flexible operation capabilities when facing different tasks and environments.

[0158] As another optional embodiment, the present invention further provides an intelligent self-loading and unloading management method, the specific execution process of which is shown in the following steps S1 to S6:

[0159] Step S1: Obtain an image of the cargo to be loaded and unloaded and a cargo loading and unloading log; perform cargo edge contour segmentation on the image of the cargo to be loaded and unloaded, and perform cargo information matching according to the cargo loading and unloading log to obtain attribute information of each cargo;

[0160] Specifically, high-definition industrial cameras are set up in the loading and unloading area to ensure that clear images of the goods can be captured. The selected camera should have a resolution of at least 1920x1080 and a frame rate of 30 frames per second to meet the monitoring needs of dynamic environments. At the same time, the camera's light source can be configured to achieve good image quality under different lighting conditions. In addition, LED (Light Emitting Diode) fill lights can be used to provide uniform lighting to avoid the impact of shadows and reflections on image quality.

[0161] Extract cargo loading and unloading logs from the warehouse management system (WMS) or cargo tracking system. The cargo loading and unloading logs contain at least information such as loading and unloading time, cargo type, quantity, weight, and location. In addition, the frequency of data extraction can be set (such as once an hour) to ensure real-time update and accuracy of the loading and unloading logs.

[0162] The collected images of the cargo to be loaded and unloaded are preprocessed, including denoising, image enhancement and grayscale conversion. This process can be achieved through Gaussian filtering and histogram equalization in the OpenCV library. At the same time, this method can improve the subsequent segmentation effect. By converting the image to a grayscale image, the subsequent edge detection process is simplified.

[0163] When performing edge contour segmentation, select a suitable contour segmentation algorithm, such as the Canny edge detection algorithm. First, apply Gaussian blur to process the image and set appropriate blur parameters (such as a 5×5 convolution kernel) to reduce the impact of noise. Set the threshold of the Canny algorithm (such as 100 and 200) to identify obvious edges in the image, so that the edge contour of the goods can be accurately captured. The findContours function in OpenCV can be used to extract the contour of the goods, and the drawContours function can be used to draw the contour on the original image for subsequent verification. The extracted contour data (such as contour coordinates and related area information) is stored in the database for subsequent processing.

[0164] Based on the information in the cargo loading and unloading log, the image of the cargo to be loaded and unloaded is matched for attribute information. Pre-set matching criteria. Including key parameters such as cargo type, size and weight. Fuzzy matching of cargo names is performed through string matching algorithms (such as Levenshtein distance), so that even slight name inconsistencies can be correctly identified. For each identified cargo outline, its corresponding information in the cargo loading and unloading log is extracted, including cargo ID, name, specifications and quantity. A matching threshold (such as 90% similarity) can be pre-set to ensure the accuracy and reliability of the matching results. Associate the attribute information of each cargo with its corresponding contour data and store it in the database to ensure the integrity of the information. Generate a visual diagram to show the outline of each cargo and its matching attribute information to help operators quickly identify and confirm the cargo.

[0165] Thus, through the image edge segmentation technology, the shape and appearance features of each cargo can be accurately extracted, eliminating the risk of misidentification. Accurate comparison with the loading and unloading log can ensure that the cargo attribute information obtained by the system is consistent with the actual cargo, thereby providing accurate basic data for subsequent operations. Specifically, through the dual verification of images and logs, the risk of human errors and missing information can be effectively reduced, ensuring that the system's identification and processing of each cargo are more intelligent and accurate. Accurate cargo attribute information provides accurate data support for loading and unloading priority assessment, path planning and collaborative control, ensuring refined management of the entire loading and unloading process.

[0166] Step S2: Perform loading and unloading risk assessment and dynamic loading and unloading priority calculation on the attribute information of each cargo, and generate the loading and unloading priority of each cargo;

[0167] Determine the key indicators for loading and unloading risk assessment, including cargo type, weight, volume, loading and unloading environment (such as temperature, humidity), historical damage records, etc. Set scoring standards for each key indicator (such as 1 to 5 points), and technical personnel in this field can score according to actual conditions. It should be pointed out that the scoring of key indicators should take into account the particularity of the cargo and environmental factors. For example, fragile goods should get high scores (such as 5 points), while durable goods should get low scores (such as 1 point). Fragile goods (such as glass) get high risk scores based on weight and type, while large goods (such as furniture) get high scores based on volume. Extract the relevant data of each piece of cargo from the cargo attribute information database to form a risk assessment data set containing all key indicators.

[0168] To ensure data accuracy, data cleaning and other techniques can be used to handle missing values ​​and outliers. For example, for missing historical damage records, mean filling or most common value filling can be used. Calculate the total risk score of each cargo based on the set scoring criteria and weights. Set a scoring threshold in advance. For example, a total risk score greater than 8 is marked as high risk, 5 to 8 is medium risk, and less than 5 is low risk. Store the calculated risk score in the database, and record the risk level (high, medium, low) and specific score of each cargo. Extract the risk score of each cargo from the risk assessment results, and combine it with the loading and unloading timeliness score to form the data set required for priority calculation.

[0169] Therefore, by comprehensively evaluating the physical characteristics of the cargo, the system can automatically identify high-risk cargo and ensure that special handling measures are taken during loading and unloading to avoid damage and accidents. As the environment, platform load and operating conditions change, the system can adjust the loading and unloading priorities in real time. Certain cargoes need to be handled first under high load conditions. The system will dynamically adjust task priorities based on real-time feedback to improve the overall operational efficiency and flexibility of the platform. Through dynamic priority sorting, the system can reasonably allocate platform resources to ensure that high-risk or urgent tasks are carried out first, avoid task accumulation and resource conflicts, and improve the overall operational fluency.

[0170] Step S3: Obtain monitoring video of the self-loading and unloading platform; perform visual recognition of the loading and unloading path on the monitoring video of the self-loading and unloading platform, perform three-dimensional topological modeling, and construct a dynamic loading and unloading model;

[0171] Install high-definition surveillance cameras around the self-loading platform. Ensure that all loading and unloading areas are covered during installation. Similar to the previous example, the installed surveillance equipment should have a resolution of at least 1920x1080 to capture clear video details. Configure the camera's shooting angle and height to ensure that the loading and unloading process can be monitored in all directions without blind spots. To capture fast-moving objects and dynamic changes, the frame rate of video recording can be set to 30 frames per second. The storage format can be selected to use H.264 encoding to save storage space and ensure that the video quality is suitable for subsequent processing.

[0172] The collected surveillance video is processed through a computer vision library (such as OpenCV). First, the video is decomposed into single-frame images. You can choose to extract 5 frames per second to reduce the processing burden. Each frame is preprocessed, including denoising, contrast enhancement, and grayscale conversion, to improve the effectiveness of subsequent edge detection and feature extraction.

[0173] Use deep learning-based object detection models (such as YOLOv5 or Faster R-CNN (Faster Region-based Convolutional Neural Network)) to identify loading and unloading paths. You can first train the model with annotated image datasets to ensure that the actual model can accurately identify paths and obstacles. Set the model's evaluation indicators (such as mAP (Mean Average Precision)), with the goal of achieving an accuracy rate of more than 80%.

[0174] On the identified loading and unloading path, key feature points (such as starting point, end point and turning point) are extracted and stored for subsequent use. At the same time, the recognition results can be saved as structured data (such as JSON format), including the coordinate information and related attributes of the path, for subsequent modeling.

[0175] Choose appropriate 3D modeling software for topology modeling. Make sure the selected tool supports importing and processing path data from computer vision. Set modeling parameters, such as the resolution and detail level of the model, to retain necessary details while ensuring performance. Create a 3D model in the modeling software based on the extracted loading and unloading path feature points. Use 3D geometric figures (such as line segments and faces) to represent the loading and unloading path. Considering the actual environment, you can import surrounding environmental data (such as shelves, robot arm positions, etc.) to ensure the authenticity and accuracy of the model.

[0176] Furthermore, the generated 3D model can be optimized to ensure smooth performance in dynamic environments. Specifically, the model can be verified through visualization tools to ensure the continuity and accuracy of the path. A virtual camera can be set up for simulation to check the correctness and operability of the model.

[0177] Define the structure of the dynamic loading and unloading model. This includes the physical model, logical model, and data model. Set the model update frequency, such as once per second, to ensure that the dynamic loading and unloading model reflects the real-time status. Integrate real-time monitoring data (such as sensor data, real-time video streams, etc.) into the dynamic loading and unloading model to achieve real-time data synchronization. In this process, communication protocols such as MQTT (Message Queuing Telemetry Transport) or WebSocket (a protocol that supports full-duplex communication) can be used for data transmission to ensure low latency and high reliability. Test the dynamic loading and unloading model to ensure that it can correctly reflect state changes under different conditions (such as high load, failure, etc.).

[0178] Therefore, through video monitoring and visual recognition, the platform can perceive changes in the working environment in real time and accurately reflect them in three-dimensional space. The construction of the twin model makes the state of the virtual platform highly synchronized with the actual platform, improving the system's ability to respond to changes in the external environment. Through three-dimensional modeling, the system can analyze the working space in real time, identify loading and unloading paths and obstacles, and provide accurate spatial data for subsequent path planning and task scheduling, reducing manual intervention and improving work safety. Based on the constructed dynamic twin model, the system continuously optimizes and adjusts the operation strategy according to real-time feedback data to ensure the smooth progress of the entire loading and unloading process. In the subsequent processing flow, the system is able to analyze the optimal loading and unloading path for each cargo according to the cargo loading and unloading priority.

[0179] Step S4: Analyze the available loading and unloading paths for each cargo in the dynamic loading and unloading model according to the loading and unloading priority of each cargo, make loading and unloading timing optimization decisions, and construct the cargo loading and unloading timing optimization path;

[0180] According to the dynamic loading and unloading model constructed above, the real situation of the loading and unloading environment can be accurately reflected, including cargo, robotic arms, and path information. First, determine the algorithm for path analysis, such as the A* algorithm or the Dijkstra algorithm. These algorithms can effectively find the shortest path and take obstacles into account. For each cargo, the selected path algorithm is used to analyze the available loading and unloading paths from the current placement position to the target placement position.

[0181] Set algorithm parameters, such as heuristic functions and path cost calculation formulas, to ensure the accuracy of path analysis. For each cargo, calculate the available loading and unloading paths one by one based on the loading and unloading priority and the current status of the cargo. The algorithm will take into account obstacles and the impact of other cargo on the path to ensure the effectiveness of the calculation. Generate the available loading and unloading paths and path costs for each cargo, and record this information for subsequent timing optimization.

[0182] Based on the analysis of available loading and unloading paths, a cargo loading and unloading sequence optimization model can be constructed. The model should consider multiple factors, including cargo loading and unloading priorities, available loading and unloading paths, loading and unloading time, cross-path conflicts, etc. Determine the optimization goal, such as minimizing overall loading and unloading time or maximizing operation efficiency, and set appropriate constraints (such as the load capacity of each robot arm and the order of operations). Select a suitable optimization algorithm, such as genetic algorithm, particle swarm optimization (PSO) or mixed-integer linear programming (MILP), to achieve optimal decision-making for loading and unloading sequence.

[0183] Select an appropriate algorithm based on the complexity and scale of the problem. Set the corresponding algorithm parameters, such as population size, number of iterations, and fitness function, to effectively optimize the cargo loading and unloading sequence. Input the available loading and unloading paths and loading and unloading priority information into the optimization model, and run the selected algorithm for timing optimization calculations. The algorithm will generate the best loading and unloading sequence and path selection to ensure that high-priority cargo is given priority with the lowest risk. During the solution process, the number of iterations is pre-set (such as a maximum of 100 iterations) to ensure that the algorithm converges to a valid solution. Based on the optimized loading and unloading sequence decision, construct the loading and unloading sequence optimization path for each cargo, and record the key nodes of the path (such as the starting point, end point, and turning point).

[0184] Generate detailed data of the loading and unloading sequence optimization path. Including the length of each path, the estimated loading and unloading time, and the information of the loading and unloading equipment passed. Verify the generated loading and unloading sequence optimization path to ensure the feasibility of the path in a dynamic environment. Specifically, a simulation tool can be used to simulate the path to test the availability and safety of the path. During the simulation process, if potential conflicts or problems are found in the path, the path or loading and unloading sequence can be adjusted in time to ensure the safety and efficiency of the overall loading and unloading process. Store the optimized loading and unloading sequence path and related information in the database to make the data traceable and complete. Record the ID, optimized path, loading and unloading sequence and estimated time of each cargo. Set the storage format to structured data (such as JSON or CSV) for subsequent analysis and query.

[0185] Through dynamic path planning, conflicts and intersections between paths are avoided, ensuring that the robot arm and other equipment can perform tasks smoothly. In addition to path planning, the system can also optimize the loading and unloading sequence to avoid resource conflicts and idle waste, ensure that tasks are performed in the optimal order, and improve the operating efficiency of the entire system.

[0186] Step S5: Control the coordinated operation of the platform manipulator according to the cargo loading and unloading timing optimization path, and perform quantitative calculation of the coordinated efficiency to generate a multi-task coordinated operation efficiency value;

[0187] First, a multi-robot collaborative control system can be designed on the self-loading and unloading platform to ensure that the robots can work together according to the optimized path. The system should at least include a central control unit, which is responsible for real-time scheduling and coordination of the actions of the robots. Control parameters, such as control frequency (such as 50Hz), are set to ensure that the system can respond to changes in the dynamic environment in real time.

[0188] Build a task scheduling mechanism to assign tasks to each robot according to the loading and unloading sequence optimization path. Use priority-based scheduling algorithms (such as EDF scheduling) to arrange loading and unloading tasks to ensure that high-priority goods are handled in a timely manner. Set scheduling parameters (such as task response time and maximum waiting time) to optimize scheduling results. Input the optimized loading and unloading sequence path into the control unit of each robot through the control system to ensure that the robot performs loading and unloading tasks according to the preset path.

[0189] Use real-time control algorithms (such as PID control) to regulate the movement of the robot arm to ensure accuracy and stability during execution. PID parameters (such as proportional, integral, and differential coefficients) can be pre-set to optimize the control effect. At the same time, sensors (such as position sensors and load sensors) are installed on each robot arm to monitor the status of the robot arm in real time. Real-time data is collected through a data acquisition system for subsequent analysis. Through real-time monitoring and control optimization, abnormal situations (such as overload or robot arm failure) can be quickly responded to and adjusted during the loading and unloading process.

[0190] Determine the evaluation indicators of multi-task collaborative operation efficiency. Including loading and unloading execution time, task completion rate, robot arm utilization rate, etc. In actual calculation, each indicator needs to be quantified for comparison and analysis. During the loading and unloading process, the loading and unloading execution time and task completion status of each robot arm are recorded in real time, and data analysis tools are used for data processing and statistics. For example, the working time, task completion status and number of conflicts of each robot arm are counted to ensure the integrity and accuracy of the data.

[0191] Therefore, by comprehensively considering multiple factors such as cargo loading and unloading priority, path planning, load status, etc., the system can complete the task in the shortest time and significantly improve the overall efficiency and accuracy of loading and unloading operations. The system can coordinate the collaborative work of multiple robotic arms in different loading and unloading tasks to avoid conflicts, waiting and invalid operations between robotic arms. Improve the collaboration ability between devices by optimizing control strategies. By calculating the work efficiency, task response time and collaborative cooperation of each robotic arm, the platform can quantify the overall efficiency of multi-task collaboration, help optimize the operation sequence, reduce invalid waiting, and improve resource utilization. The quantification of collaborative efficiency provides data basis for subsequent optimization. The system can adjust the operation strategy in real time according to different task loads and efficiency values ​​to further improve the overall collaborative efficiency of the system.

[0192] Step S6: Predict loading and unloading failures based on real-time parameters of multi-task collaborative control, and iteratively optimize dynamic loading and unloading management based on multi-task collaborative operation efficiency values ​​to build an intelligent loading and unloading management model.

[0193] On the self-loading and unloading platform, real-time control parameters of multi-task collaborative control are continuously collected, such as the motion state of the robot arm, load, temperature, humidity, etc. These data can be collected in real time through sensors and monitoring systems to ensure the accuracy and timeliness of the data. Set the data collection frequency, such as once per second, to ensure real-time monitoring of the system status.

[0194] Select a suitable fault prediction model, such as a machine learning-based classification model (such as random forest, support vector machine) or a deep learning model (such as long short-term memory network) for time series prediction. These models can process time series data and effectively identify potential failure modes. In the model training phase, historical fault data and real-time parameters can be used for training to improve the model's prediction ability. Set the ratio of training set and test set according to actual needs (such as 80% training and 20% testing) to ensure the generalization ability of the model. Input the collected real-time parameters into the trained fault prediction model to evaluate the failure risk of the loading and unloading equipment in real time. A prediction threshold can be set (such as marking it as high risk when the probability of failure is greater than 0.7) so that timely measures can be taken. Output the prediction results, including potential fault types, probability of occurrence, and recommended preventive measures. Store the prediction results in a database for subsequent analysis and decision-making.

[0195] By real-time monitoring and analyzing the platform's operating data, the system can predict failures that occur during loading and unloading, and take measures in advance to avoid failures. When abnormal load or path conflict is detected in the robotic arm, the system can immediately adjust the task or notify the operator to intervene. Based on real-time feedback on the multi-task collaborative efficiency value, the system can continuously optimize the loading and unloading strategy and make adjustments based on the actual operating conditions to improve the overall operating efficiency of the system.

[0196] On the basis of fault prediction, considering the fault prediction results, real-time parameters and multi-task collaborative operation efficiency values, setting optimization goals (such as minimizing the fault occurrence rate and loading and unloading time), and building a dynamic loading and unloading management iterative optimization model. According to actual needs, select appropriate optimization algorithms, such as genetic algorithms or reinforcement learning, to ensure that the model can make effective decisions in a dynamic environment. At the same time, it is necessary to determine the key parameters of the optimization model, including the weight of fault prediction, the weight of loading and unloading efficiency value, and the system response time. Among them, these parameters can be obtained through expert evaluation or historical data analysis.

[0197] Set the evaluation indicators of the model, such as the percentage increase in loading and unloading efficiency and the percentage reduction in failure rate after optimization, for iterative evaluation. During the actual loading and unloading process, run the iterative optimization model in real time and make dynamic adjustments based on real-time data and failure prediction results. The update frequency of the model can be set in advance (such as once a minute) to ensure that the model adapts to the rapidly changing environment. By setting up a feedback mechanism, collect the results after each optimization, evaluate the loading and unloading efficiency and the occurrence of failures, and facilitate subsequent iterations. At the same time, analyze the results of each iteration to determine whether the optimization effect has achieved the expected goal. If not, it is necessary to dynamically adjust the model parameters and optimization strategy.

[0198] Through continuous learning and optimization, the system can gradually improve the intelligence and flexibility of decision-making. Through intelligent fault prediction, real-time monitoring and self-optimization, the system can maintain efficient, stable and reliable operation, reduce downtime, improve the continuity of loading and unloading operations and the stability of the system.

[0199] Adjust the loading and unloading management strategy based on the analysis results. For example, adjust the operation sequence of the robot arm, optimize the loading and unloading path, etc. to improve the overall efficiency. In this way, the fault prediction and dynamic loading and unloading management iterative optimization model are integrated into an intelligent loading and unloading management model, so that the system can make better real-time decisions and management of cargo loading and unloading. Ensure that the model can seamlessly connect real-time data and historical data. Finally, deploy the intelligent loading and unloading management model to the central control system of the self-loading and unloading platform to ensure that the system can make dynamic decisions based on real-time parameters and prediction results.

[0200] Reference Figure 8 , shows a structural block diagram of an intelligent self-loading and unloading management device provided by an embodiment of the present invention, which may specifically include:

[0201] The data acquisition unit 801 is used to acquire images of cargo to be loaded and unloaded, cargo loading and unloading logs, and monitoring videos of the loading and unloading platform;

[0202] A cargo information matching unit 802 is used to perform cargo information matching based on edge recognition according to the cargo image to be loaded and unloaded and the cargo loading and unloading log to obtain cargo attribute information of each cargo;

[0203] A loading and unloading priority generation unit 803, configured to perform a dynamic loading and unloading priority calculation based on loading and unloading risk assessment in combination with the cargo loading and unloading log and each cargo attribute information, and generate a loading and unloading priority for each cargo;

[0204] A dynamic loading and unloading model building unit 804 is used to identify the loading and unloading path of the monitoring video and build a dynamic loading and unloading model;

[0205] The loading and unloading path planning unit 805 is used to analyze available loading and unloading paths according to the dynamic loading and unloading model, and make loading and unloading timing optimization decisions based on each of the loading and unloading priorities to construct a timing optimization path for cargo loading and unloading;

[0206] The intelligent loading and unloading management unit 806 is used to perform multi-task collaborative control calculations based on the timing optimization path, obtain real-time control parameters and operational efficiency values ​​of multi-task collaboration, and perform dynamic loading and unloading management optimization based on the real-time control parameters and the operational efficiency values, and construct an intelligent loading and unloading management model. The intelligent loading and unloading management model is used to make real-time decisions and management on cargo loading and unloading.

[0207] In an optional embodiment, the cargo information matching unit 802 includes:

[0208] A cargo edge contour segmentation unit, used to segment the cargo edge contour of the cargo image to be loaded and unloaded, and extract the edge contour line of each cargo;

[0209] A geometric analysis unit, used to perform geometric analysis on each edge contour line to generate geometric features of each cargo;

[0210] A cargo surface texture recognition unit, used to perform cargo surface texture recognition on the cargo image to be loaded and unloaded, and extract cargo surface texture features;

[0211] A deep visual recognition unit, used to perform deep visual recognition based on each of the geometric features and the surface texture features of the goods, respectively, to generate a deep visual representation of each of the goods;

[0212] The cargo information matching subunit is used to match the cargo information of the cargo loading and unloading log according to each of the deep visual representations to obtain cargo attribute information of each cargo.

[0213] In an optional embodiment, the loading and unloading priority generation unit 803 includes:

[0214] A loading and unloading risk assessment unit, used to perform loading and unloading risk assessment on each of the cargo attribute information, and generate a loading and unloading risk assessment value for each cargo;

[0215] A cargo flow mining unit, used for mining cargo flow based on the cargo loading and unloading logs to generate flow data for each cargo;

[0216] A final placement position calculation unit, used to calculate the final placement position according to each of the flow direction data, and generate the final placement position of each of the goods;

[0217] A real-time platform loading and unloading load analysis unit, used to perform real-time platform loading and unloading load analysis according to the cargo loading and unloading log to obtain real-time loading and unloading load data;

[0218] The dynamic loading and unloading priority calculation unit is used to perform dynamic loading and unloading priority calculation on the loading and unloading risk assessment value and the final placement position of each cargo according to the real-time loading and unloading load data, and generate the loading and unloading priority of each cargo.

[0219] In an optional embodiment, the dynamic loading and unloading model building unit 804 includes:

[0220] A loading and unloading path visual recognition unit, used to perform loading and unloading path visual recognition on the monitoring video to obtain multiple loading and unloading paths;

[0221] A platform component analysis unit, used to perform platform component analysis on the monitoring video to obtain multiple platform components;

[0222] A path obstacle monitoring unit, configured to perform path obstacle monitoring according to the plurality of platform components and mark each path obstacle node;

[0223] A spatial position positioning unit, used to perform spatial position positioning on each of the path obstacle nodes to generate spatial position coordinates;

[0224] The three-dimensional topological modeling unit is used to perform three-dimensional topological modeling based on the multiple loading and unloading paths and each of the spatial position coordinates to construct a dynamic loading and unloading model.

[0225] In an optional embodiment, the three-dimensional topology modeling unit includes:

[0226] A current cargo placement position identification unit, used to identify the current cargo placement position;

[0227] A platform operation space calculation unit, used to calculate the platform operation space based on the current cargo placement position and each of the spatial position coordinates to obtain platform operation space characteristics;

[0228] A platform component spatial layout analysis unit, configured to perform platform component spatial layout analysis on the plurality of platform components to generate spatial layout data of the platform components;

[0229] A platform three-dimensional topological structure analysis unit, used to perform a platform three-dimensional topological structure analysis based on the spatial layout data to obtain three-dimensional topological structure features;

[0230] A loading and unloading robot arm state parameter acquisition unit, used to acquire the loading and unloading robot arm state parameters of the self-loading and unloading platform;

[0231] An operation status analysis unit, used to perform operation status analysis on the state parameters of the loading and unloading robot arm to obtain operation status characteristics of the robot arm;

[0232] A robot arm loading and unloading activity range calculation unit, used to calculate the robot arm loading and unloading activity range according to the robot arm operation state characteristics, and extract the robot arm loading and unloading activity range;

[0233] A three-dimensional point cloud modeling unit, used to perform three-dimensional point cloud modeling on the three-dimensional topological structure features according to the platform operation space features, and construct a three-dimensional loading and unloading platform model;

[0234] The dynamic loading and unloading behavior rendering unit is used to render the dynamic loading and unloading behavior of the three-dimensional loading and unloading platform model according to the loading and unloading activity range of the robot arm and the multiple loading and unloading paths, so as to construct a dynamic loading and unloading model.

[0235] In an optional embodiment, the loading and unloading path planning unit 805 includes:

[0236] An available loading and unloading path analysis unit, used to analyze the available loading and unloading paths for each cargo one by one according to the dynamic loading and unloading model, and generate an available loading and unloading path for each cargo;

[0237] A cargo transportation intersection prediction unit, used to perform cargo transportation intersection prediction on each of the available loading and unloading paths, and mark the loading and unloading path intersections;

[0238] A potential path obstacle detection unit, used to perform potential path obstacle detection on each of the available loading and unloading paths to identify potential path obstacles;

[0239] An available path avoidance analysis unit, used to perform available path avoidance analysis according to the potential path obstacles and the intersection of the loading and unloading paths, and generate an avoidance path for each cargo;

[0240] The loading and unloading timing optimization decision unit is used to make loading and unloading timing optimization decisions for each of the avoidance paths according to each of the loading and unloading priorities, and construct a cargo loading and unloading timing optimization path.

[0241] In an optional embodiment, the intelligent loading and unloading management unit 806 includes:

[0242] A real-time control parameter acquisition unit, used to perform collaborative operation control of the platform manipulator according to the timing optimization path and to acquire real-time control parameters of multi-task collaboration;

[0243] A loading and unloading execution time calculation unit, used to calculate the loading and unloading execution time for the real-time control parameters to generate the loading and unloading execution time for each cargo;

[0244] A loading and unloading conflict statistics unit, used to obtain multi-task collaborative operation simulation data, and to perform loading and unloading conflict statistics on the operation simulation data, and to identify loading and unloading conflict time points;

[0245] A conflict path distribution mining unit is used to perform conflict path distribution mining on the loading and unloading conflict time points to obtain loading and unloading conflict path distribution data;

[0246] The collaborative efficiency quantification calculation unit is used to perform collaborative efficiency quantification calculation based on the loading and unloading conflict path distribution data and each loading and unloading execution time to generate an operational efficiency value of multi-task collaboration.

[0247] In an optional embodiment, the intelligent loading and unloading management unit 806 includes:

[0248] A loading and unloading fault prediction unit, used to predict loading and unloading faults according to the real-time control parameters and generate loading and unloading fault prediction data;

[0249] A fault tracing and locating unit, used to perform fault tracing and locating on the loading and unloading fault prediction data to obtain a fault tracing point of the self-loading and unloading platform;

[0250] An instant fault diagnosis unit, used to perform instant fault diagnosis on the fault tracing point to obtain an instant fault diagnosis result;

[0251] The dynamic loading and unloading management iterative optimization unit is used to perform dynamic loading and unloading management iterative optimization based on the operation efficiency value and the real-time fault diagnosis result to build an intelligent loading and unloading management model.

[0252] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the aforementioned method embodiment.

[0253] An embodiment of the present invention further provides an electronic device, the device comprising a processor and a memory:

[0254] The memory is used to store program codes and transmit the program codes to the processor; the processor is used to execute the intelligent self-loading and unloading management method of any embodiment of the present invention according to the instructions in the program codes.

[0255] An embodiment of the present invention further provides a computer-readable storage medium, which is used to store program codes, and the program codes are used to execute the intelligent self-loading and unloading management method of any embodiment of the present invention.

[0256] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0257] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0258] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0259] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0260] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0261] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent self-loading and unloading management method, characterized in that: include: Obtain images of cargo to be loaded and unloaded, cargo loading and unloading logs, and monitoring videos of self-loading and unloading platforms; Cargo information matching based on edge recognition is performed according to the cargo image to be loaded and unloaded and the cargo loading and unloading log to obtain cargo attribute information of each cargo; Combine the cargo loading and unloading log and the attribute information of each cargo to perform dynamic loading and unloading priority calculation based on loading and unloading risk assessment, and generate the loading and unloading priority of each cargo; Perform loading and unloading path recognition on the monitoring video and construct a dynamic loading and unloading model; Analyze available loading and unloading paths according to the dynamic loading and unloading model, make loading and unloading timing optimization decisions based on each of the loading and unloading priorities, and construct a timing optimization path for cargo loading and unloading; Based on the timing optimization path, multi-task collaborative control calculations are performed to obtain real-time control parameters and operational efficiency values ​​of multi-task collaboration, and dynamic loading and unloading management optimization is performed based on the real-time control parameters and the operational efficiency values ​​to construct an intelligent loading and unloading management model. The intelligent loading and unloading management model is used to make real-time decisions and management on cargo loading and unloading.

2. The intelligent self-loading and unloading management method according to claim 1 is characterized in that: The performing edge recognition-based cargo information matching according to the cargo image to be loaded and unloaded and the cargo loading and unloading log to obtain cargo attribute information of each cargo includes: Performing cargo edge contour segmentation on the cargo image to be loaded and unloaded, and extracting the edge contour line of each cargo; Performing geometric analysis on each edge contour line to generate geometric features of each cargo; Performing cargo surface texture recognition on the cargo to be loaded and unloaded image to extract cargo surface texture features; Performing deep visual recognition based on each of the geometric features and the surface texture features of the goods to generate a deep visual representation of each of the goods; Cargo information is matched with the cargo loading and unloading log according to each of the deep visual representations to obtain cargo attribute information of each cargo.

3. The intelligent self-loading and unloading management method according to claim 1 is characterized in that: The step of performing a dynamic loading and unloading priority calculation based on loading and unloading risk assessment in combination with the cargo loading and unloading log and the attribute information of each cargo to generate the loading and unloading priority of each cargo includes: Performing loading and unloading risk assessment on each cargo attribute information respectively to generate a loading and unloading risk assessment value for each cargo; Cargo flow mining is performed based on the cargo loading and unloading logs to generate flow data for each cargo; Calculate the final placement position according to each of the flow direction data to generate the final placement position of each cargo; Perform real-time platform loading and unloading load analysis according to the cargo loading and unloading log to obtain real-time loading and unloading load data; The loading and unloading risk assessment value and the final placement position of each cargo are dynamically calculated based on the real-time loading and unloading load data to generate the loading and unloading priority of each cargo.

4. The intelligent self-loading and unloading management method according to claim 1 is characterized in that: The step of identifying the loading and unloading path of the monitoring video and constructing a dynamic loading and unloading model includes: Performing visual recognition of loading and unloading paths on the monitoring video to obtain multiple loading and unloading paths; Performing platform component analysis on the monitoring video to obtain multiple platform components; Performing path obstacle monitoring according to the plurality of platform components and marking each path obstacle node; Positioning each of the path obstacle nodes in space to generate spatial position coordinates; Three-dimensional topological modeling is performed based on the multiple loading and unloading paths and each of the spatial position coordinates to construct a dynamic loading and unloading model.

5. The intelligent self-loading and unloading management method according to claim 4 is characterized in that: The three-dimensional topological modeling is performed based on the multiple loading and unloading paths and each of the spatial position coordinates to construct a dynamic loading and unloading model, including: Identify the current location of goods; Calculate the platform operation space based on the current cargo placement position and each of the spatial position coordinates to obtain platform operation space characteristics; Performing platform component spatial layout analysis on the multiple platform components to generate spatial layout data of the platform components; Performing a three-dimensional topological structure analysis of the platform based on the spatial layout data to obtain three-dimensional topological structure features; Obtaining state parameters of the loading and unloading robot arm of the self-loading and unloading platform; Performing an operation status analysis on the state parameters of the loading and unloading robot arm to obtain the operation status characteristics of the robot arm; Calculate the loading and unloading range of the robot arm according to the operating state characteristics of the robot arm, and extract the loading and unloading range of the robot arm; Performing 3D point cloud modeling on the 3D topological structure features according to the platform operation space features to construct a 3D loading and unloading platform model; According to the loading and unloading activity range of the robot arm and the multiple loading and unloading paths, the three-dimensional loading and unloading platform model is dynamically rendered to construct a dynamic loading and unloading model.

6. The intelligent self-loading and unloading management method according to claim 1 or 3, characterized in that: The analyzing available loading and unloading paths according to the dynamic loading and unloading model, making loading and unloading timing optimization decisions based on each of the loading and unloading priorities, and constructing a timing optimization path for cargo loading and unloading includes: Analyzing available loading and unloading paths for each cargo one by one according to the dynamic loading and unloading model, and generating available loading and unloading paths for each cargo; Cargo transportation intersection prediction is performed for each of the available loading and unloading paths, and the loading and unloading path intersection points are marked; Performing potential path obstacle detection on each of the available loading and unloading paths to identify potential path obstacles; Performing available path avoidance analysis based on the potential path obstacles and the loading and unloading path intersections to generate an avoidance path for each cargo; A loading and unloading timing optimization decision is made for each of the avoidance paths according to each of the loading and unloading priorities, and a cargo loading and unloading timing optimization path is constructed.

7. The intelligent self-loading and unloading management method according to claim 1 is characterized in that: The multi-task collaborative control calculation is performed based on the timing optimization path to obtain the real-time control parameters and operation efficiency value of the multi-task collaboration, including: Perform collaborative operation control of the platform robot arm according to the timing optimization path, and collect real-time control parameters of multi-task collaboration; Calculating the loading and unloading execution time for the real-time control parameters to generate the loading and unloading execution time for each cargo; Acquire multi-task collaborative operation simulation data, perform loading and unloading conflict statistics on the operation simulation data, and identify loading and unloading conflict time points; Mining the conflict path distribution of the loading and unloading conflict time points to obtain the loading and unloading conflict path distribution data; Based on the loading and unloading conflict path distribution data and each loading and unloading execution time, the collaborative efficiency is quantitatively calculated to generate an operational efficiency value of multi-task collaboration.

8. The intelligent self-loading and unloading management method according to claim 1 or 7, characterized in that: The method of performing dynamic loading and unloading management optimization according to the real-time control parameters and the operation efficiency value and constructing an intelligent loading and unloading management model includes: Perform loading and unloading fault prediction according to the real-time control parameters to generate loading and unloading fault prediction data; Performing fault tracing and locating on the loading and unloading fault prediction data to obtain the fault tracing point of the self-loading and unloading platform; Performing instant fault diagnosis on the fault tracing point to obtain an instant fault diagnosis result; Based on the operation efficiency value and the instant fault diagnosis result, dynamic loading and unloading management is iteratively optimized to build an intelligent loading and unloading management model.

9. An intelligent self-loading and unloading management device, characterized in that: include: A data acquisition unit, used to acquire images of cargo to be loaded and unloaded, cargo loading and unloading logs, and monitoring videos of the loading and unloading platform; A cargo information matching unit, configured to perform cargo information matching based on edge recognition according to the cargo image to be loaded and unloaded and the cargo loading and unloading log, to obtain cargo attribute information of each cargo; A loading and unloading priority generation unit, configured to perform a dynamic loading and unloading priority calculation based on loading and unloading risk assessment in combination with the cargo loading and unloading log and each cargo attribute information, and generate a loading and unloading priority for each cargo; A dynamic loading and unloading model building unit, used to identify the loading and unloading path of the monitoring video and build a dynamic loading and unloading model; A loading and unloading path planning unit, used to analyze available loading and unloading paths according to the dynamic loading and unloading model, and make loading and unloading timing optimization decisions based on each of the loading and unloading priorities to construct a timing optimization path for cargo loading and unloading; The intelligent loading and unloading management unit is used to perform multi-task collaborative control calculations based on the timing optimization path, obtain real-time control parameters and operational efficiency values ​​of multi-task collaboration, and perform dynamic loading and unloading management optimization based on the real-time control parameters and the operational efficiency values, and construct an intelligent loading and unloading management model. The intelligent loading and unloading management model is used to make real-time decisions and management on cargo loading and unloading.

10. An electronic device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the intelligent self-loading and unloading management method according to any one of claims 1 to 7 according to the instructions in the program code.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the intelligent self-loading and unloading management method according to any one of claims 1 to 7.

Images