Resource scheduling method and system for warehouse video analysis based on cloud-edge collaboration

Abstract

The invention relates to the technical field of resource scheduling, and discloses a warehouse video analysis resource scheduling method and system based on cloud edge collaboration, and the method comprises the steps: obtaining the monitoring video stream of each camera of a warehouse, determining the monitoring region of each camera, dividing the region priority of all the monitoring regions according to the goods storage requirements, and carrying out the regional priority of each camera; and based on the target frame image and the convolutional neural network model, determining whether there is a goods placement error in the corresponding monitoring area, if there is a goods placement error, sending out a corresponding early warning according to the area priority of the corresponding monitoring area, and if there is no goods placement error, sending out a corresponding early warning according to the area priority of the corresponding monitoring area. And determining an edge processing strategy or a cloud processing strategy based on the corresponding region priority, and adjusting resource allocation of the cloud server based on the migration complexity. According to the invention, the operation efficiency and reliability of warehouse monitoring are ensured, so that the balance between the efficiency and stability of resource scheduling is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of resource scheduling, and in particular to a resource scheduling method and system based on cloud-edge collaboration for warehouse video analysis. Background Art

[0002] In warehouse management, a certain number of cameras are deployed to achieve real-time monitoring of the storage status of goods. However, as warehouses expand in size and the variety of goods increases, video data is exploding. Currently, video analysis and processing primarily utilizes either centralized cloud processing or decentralized edge device processing. While cloud processing offers powerful computing power, video streaming consumes a large amount of bandwidth and suffers from high network latency, making it difficult to meet real-time requirements. While edge processing can reduce latency and bandwidth consumption, edge devices have limited computing power and cannot efficiently handle complex video analysis tasks. Furthermore, existing resource utilization fails to fully consider the priority differences between different warehouse areas. Furthermore, during task migration, cloud server resource allocation cannot be flexibly adjusted based on the load data volume and migration complexity, impacting the operational efficiency and reliability of warehouse monitoring. Therefore, it is necessary to design a resource scheduling method and system for warehouse video analysis based on cloud-edge collaboration to solve the problems existing in current technology. Summary of the Invention

[0003] In view of this, the present invention proposes a resource scheduling method and system for warehouse video analysis based on cloud-edge collaboration, aiming to solve the above problems.

[0004] In one aspect, the present invention proposes a resource scheduling method for warehouse video analysis based on cloud-edge collaboration, comprising: Obtain surveillance video streams from all cameras in the warehouse and determine the monitoring area of ​​each camera, and prioritize all monitoring areas based on cargo storage requirements; Extracting the surveillance video stream of each monitoring area to determine the current frame image, preprocessing the current frame image to determine the target frame image, and determining whether there is any cargo placement error in the corresponding monitoring area based on the target frame image and the convolutional neural network model; When there is a cargo placement error, a corresponding warning is issued based on the regional priority of the corresponding monitoring area. When there is no cargo placement error, the edge processing strategy or cloud processing strategy is determined based on the corresponding regional priority. The load data volume is determined based on the edge processing strategy or the cloud processing strategy, and the edge processing tasks in the edge processing strategy are migrated to the cloud server according to the load data volume, and the resource allocation of the cloud server is adjusted based on the migration complexity.

[0005] Furthermore, when all monitoring areas are divided into regional priorities according to cargo storage requirements, they include: determining a complexity score based on cargo storage requirements, wherein the complexity score is positively correlated with the cargo storage requirements; determining a first complexity score, a second complexity score, and a third complexity score, wherein the first complexity score is greater than the second complexity score, and the second complexity score is greater than the third complexity score; When the complexity score is greater than or equal to the first complexity score, the corresponding monitoring area is determined as a first-level priority; When the complexity score is less than the first complexity score and greater than the second complexity score, the corresponding monitoring area is determined as a second-level priority; When the complexity score is less than or equal to the second complexity score, the corresponding monitoring area is determined to be a third level priority; The resource inputs for the first-level priority, the second-level priority, and the third-level priority decrease in sequence.

[0006] Furthermore, when preprocessing the current frame image to determine the target frame image, the method includes: The current frame image is subjected to wavelet decomposition and noise processing based on BayesShrink soft shrinkage, and the noise processing result is subjected to limited contrast adaptive histogram equalization, and the cropping intensity is adjusted according to the image entropy after noise processing to determine the target frame image.

[0007] Furthermore, when determining whether there is a cargo placement error in the corresponding monitoring area based on the target frame image and the convolutional neural network model, the method includes: Based on grid search, model parameters of a convolutional neural network model are found, and a convolutional neural network model is established. The convolutional neural network model includes a convolution layer, a pooling layer, and a fully connected layer, and a binary classification output layer is provided in the fully connected layer; Obtaining a sample data set and dividing the sample data set into a training set and a test set, training the convolutional neural network model based on the training set, and substituting the test set into the trained convolutional neural network model to determine the prediction accuracy; If the prediction accuracy of the currently trained convolutional neural network model is greater than or equal to the prediction accuracy of the previously trained convolutional neural network model, and the prediction accuracy of the currently trained convolutional neural network model is greater than 0.85, then training is stopped. Otherwise, different network layers are differentially adjusted according to L1 and L2 regularization, and training is continued. The target frame image is substituted into the convolutional neural network model after stopping training to determine whether there is any error in the placement of goods in the corresponding monitoring area.

[0008] Furthermore, when there is a misplacement of goods, corresponding warnings are issued based on the regional priority of the corresponding monitoring area, including: When the corresponding monitoring area is at the first-level priority level, a red alert is triggered; When the corresponding monitoring area is at the second-level priority level, an orange warning is triggered; When the corresponding monitoring area is at the third level of priority, a yellow warning is triggered; When a red warning / orange warning / yellow warning is triggered, the time it takes to trigger the warning is positively correlated with the number of incorrectly placed goods.

[0009] Furthermore, when there are no cargo placement errors, the edge processing strategy or cloud processing strategy is determined based on the corresponding regional priority, including: Determine the first-level priority, second-level priority, and third-level priority real-time anomaly detection tasks as the edge processing strategy, wherein the edge processing strategy includes cargo stacking overrun detection and personnel illegal entry detection for edge devices; The trend analysis tasks and cargo tracing tasks of the first-level priority, the second-level priority and the third-level priority are determined as the cloud processing strategy, and the cloud processing strategy is executed by the cloud server including inventory change trend analysis and abnormal event history backtracking.

[0010] Furthermore, when determining the load data volume based on the edge processing strategy or the cloud processing strategy, it includes: The load data volume of the edge processing strategy includes the current frame image extraction data volume and the number of concurrent tasks; The load data volume of the cloud processing strategy includes the cloud transmission data volume and the new task data transmission data volume.

[0011] Furthermore, when migrating the edge processing tasks in the edge processing strategy to the cloud server according to the load data volume, the method includes: Recording the load data volume of the edge processing strategy as edge load data volume, and comparing the edge load data volume with an edge load data volume threshold, where the edge load data volume threshold is determined according to the rated computing power of the edge device; When the edge load data volume is greater than or equal to the edge load data volume threshold, the tasks are prioritized for migration. The priority is determined based on the real-time performance of the tasks, the data encryption level, and the migration bandwidth consumption. The real-time coefficient and the data encryption level are positively correlated with the cargo storage requirements. Based on the results of task migration priority sorting, edge processing tasks are migrated from high to low, and the migration process adopts incremental data transmission.

[0012] Furthermore, when adjusting resource allocation of the cloud server based on the migration complexity, it includes: The migration complexity is determined based on the data transmission delay and the conflict rate of similar tasks on the cloud. The conflict rate of similar tasks on the cloud is the overlap ratio between the computing power resources required for the current migrating edge processing task and the resources allocated on the cloud. The resource allocation weight of the regional priority in the cloud server is adjusted based on the migration complexity.

[0013] Compared with the existing technology, the beneficial effects of the present invention are as follows: by dividing the regional priorities of the monitoring areas, it realizes the differentiated and precise allocation of resources, avoids the risk of monitoring delays or missed detections due to resource preemption in key areas, improves the pertinence and reliability of warehouse monitoring, and effectively balances the requirements of real-time performance and computing power according to the different processing strategies of edge devices and cloud servers, making up for the limitations of a single processing mode. The task migration mechanism based on the load data volume can dynamically match processing requirements and resource capabilities. It avoids overloading of edge devices or idle cloud resources, thereby improving the overall resource utilization and ensuring the efficient operation of video analysis tasks. It dynamically adjusts the resource allocation of cloud servers based on the complexity of migration, thereby optimizing resource configuration in a targeted manner, adapting more resources to relatively complex migration tasks to ensure processing efficiency; and streamlining resources for simple tasks to reduce operating costs, thereby achieving a balance between the efficiency and stability of resource scheduling.

[0014] On the other hand, the present application also provides a resource scheduling system based on cloud-edge collaborative warehouse video analysis, which is used to apply the above-mentioned resource scheduling method based on cloud-edge collaborative warehouse video analysis, including: The acquisition and analysis unit is configured to obtain the surveillance video streams of each camera in the warehouse and determine the monitoring area of ​​each camera, and divide all the monitoring areas into regional priorities according to the storage requirements of the goods; a judgment processing unit configured to extract the surveillance video stream of each surveillance area to determine a current frame image, preprocess the current frame image to determine a target frame image, and determine whether there is a cargo placement error in the corresponding surveillance area based on the target frame image and a convolutional neural network model; a strategy determination unit configured to, when a cargo placement error occurs, issue a corresponding warning based on the regional priority of the corresponding monitoring area, and, when no cargo placement error occurs, determine an edge processing strategy or a cloud processing strategy based on the corresponding regional priority; The resource scheduling unit is configured to determine the load data volume based on the edge processing strategy or the cloud processing strategy, migrate the edge processing tasks in the edge processing strategy to the cloud server according to the load data volume, and adjust the resource allocation of the cloud server based on the migration complexity.

[0015] It can be understood that the above-mentioned resource scheduling method and system based on cloud-edge collaboration warehouse video analysis have the same beneficial effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings: Figure 1 A flowchart of a resource scheduling method for warehouse video analysis based on cloud-edge collaboration provided by an embodiment of the present invention; Figure 2 Functional block diagram of the resource scheduling system for warehouse video analysis based on cloud-edge collaboration provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, unless there is a conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0018] In some embodiments of the present application, see Figure 1 As shown in FIG, a resource scheduling method for warehouse video analysis based on cloud-edge collaboration includes: S100: Obtain the surveillance video streams of each camera in the warehouse and determine the surveillance area of ​​each camera, and divide all surveillance areas into regional priorities according to the cargo storage requirements.

[0019] S200: Extract the surveillance video stream of each surveillance area to determine the current frame image, preprocess the current frame image to determine the target frame image, and determine whether there is any cargo placement error in the corresponding surveillance area based on the target frame image and the convolutional neural network model.

[0020] S300: When there is a cargo placement error, a corresponding warning is issued according to the regional priority of the corresponding monitoring area. When there is no cargo placement error, an edge processing strategy or a cloud processing strategy is determined based on the corresponding regional priority.

[0021] S400: Determine the load data volume based on the edge processing strategy or the cloud processing strategy, migrate the edge processing tasks in the edge processing strategy to the cloud server according to the load data volume, and adjust the resource allocation of the cloud server based on the migration complexity.

[0022] Specifically, traditional cloud-based centralized processing is limited by bandwidth and latency, making it unable to meet the real-time warehouse monitoring needs. Pure edge processing, however, is limited by the computing power of the devices and struggles to cope with complex analytical tasks. Furthermore, it ignores differences in regional priorities, leading to unbalanced resource allocation. The lack of a dynamic resource adjustment mechanism during task migration further impacts the operational efficiency and reliability of warehouse monitoring. Multiple cameras deployed throughout the warehouse collect real-time video streams from each monitoring area. Typically, a monitoring area is responsible for monitoring a specific area of ​​the warehouse, such as a shelf or an independent storage compartment. Regional priorities are assigned based on cargo storage requirements (such as cargo value, turnover rate, and security level). For example, monitoring areas storing valuables or dangerous goods are assigned a higher priority, while those storing ordinary cargo are assigned a medium or lower priority. This differentiated division lays the foundation for subsequent differentiated resource allocation. Frame extraction is performed on the video stream of each monitoring area. Due to problems such as camera jitter and dust obstruction in the warehouse environment, noise and other conditions may occur during the extraction process, which directly affects the subsequent recognition accuracy. Therefore, preprocessing is required to eliminate the environmental noise and then determine the target frame image. The convolutional neural network model is used to extract features and identify the location of goods in the target frame image to determine whether there are any placement errors (such as exceeding the location boundary, etc.). The convolutional neural network model is capable of adapting to the computing power of edge devices. Edge devices are computing devices deployed in the warehouse near the camera monitoring area, including edge servers and embedded systems. When a cargo placement error is detected, different levels of alerts are triggered based on regional priority, improving the accuracy and timeliness of abnormal responses. Edge or cloud processing strategies are determined based on regional priority, leveraging the real-time processing capabilities of edge devices (such as changes in cargo stacking) and in-depth cloud analysis (such as long-term trend prediction) to process different tasks. This balances real-time performance with computing power consumption, avoiding the risk of delays for higher-priority regional tasks due to insufficient edge computing power. After determining the corresponding processing strategy, the edge device's load data volume is quantified. Due to the varying computing resource requirements of different tasks, some tasks are automatically migrated to the cloud based on the load data volume. Cloud resource allocation is dynamically adjusted during the migration process. For example, more complex tasks with higher regional priorities are allocated more GPU computing power and storage resources. This flexible adjustment avoids wasting or overloading cloud resources. Differentiated warehouse management is achieved through regional priority division. Cloud-edge collaboration balances computing power and real-time performance, ensuring the efficiency of corresponding task processing and the reliability of warehouse monitoring operations.

[0023] It is understandable that resource scheduling and allocation are achieved through the division of regional priorities, thereby reducing bandwidth consumption. The cloud focuses on complex task processing, while the localized processing tasks of edge devices ensure the completion of basic monitoring tasks during network fluctuations, thereby improving resource utilization and reducing the risk of corresponding task failure due to resource overload.

[0024] In some embodiments of the present application, when all monitoring areas are divided into regional priorities according to cargo storage requirements, it includes: determining a complexity score according to the cargo storage requirements, the complexity score is positively correlated with the cargo storage requirements, determining a first complexity score, a second complexity score and a third complexity score, the first complexity score is greater than the second complexity score, the second complexity score is greater than the third complexity score, when the complexity score is greater than or equal to the first complexity score, the corresponding monitoring area is determined as a first-level priority, when the complexity score is less than the first complexity score and greater than the second complexity score, the corresponding monitoring area is determined as a second-level priority, when the complexity score is less than or equal to the second complexity score, the corresponding monitoring area is determined as a third-level priority, and the resource input of the first-level priority, the second-level priority and the third-level priority decreases in turn.

[0025] Specifically, the complexity score is determined by integrating multiple metrics related to cargo storage requirements, including cargo value (such as valuable electronics), storage environment sensitivity (such as pharmaceutical reagents requiring constant temperature and humidity), safety risk level (such as flammable and explosive hazardous materials), and turnover rate (such as fast-moving consumer goods with frequent in-and-out storage). Each metric is weighted and cumulatively added to create a complexity score. A higher complexity score indicates more stringent storage requirements. The first, second, and third complexity scores are calibrated based on actual warehouse operational data. For example, areas requiring safety monitoring (such as chemical storage) are classified as having a complexity score of first or higher, general industrial parts storage areas are classified between the first and second complexity scores, and temporary storage areas such as slow-moving goods are classified as having a complexity score of third or lower. This transforms vague storage requirements into actionable priority criteria, enabling efficient and quantitative resource allocation while avoiding the subjectivity of human judgment. This allows first-level priority tasks to receive more computing power from edge devices and faster cloud response channels, thereby improving resource utilization while avoiding congestion caused by lower-priority tasks occupying resources.

[0026] In some embodiments of the present application, when pre-processing the current frame image to determine the target frame image, it includes: using wavelet decomposition on the current frame image and performing noise processing based on BayesShrink soft shrinkage, and performing limited contrast adaptive histogram equalization on the result of the noise processing, and adjusting the cropping intensity according to the image entropy after noise processing to determine the target frame image.

[0027] Specifically, wavelet decomposition decomposes the current frame image into a low-frequency approximate component (containing image information) and a high-frequency detail component (containing some noise). Bayes Shrink soft shrinkage uses adaptive calculations to reduce noise in the high-frequency detail component, preserving key details such as product edges while suppressing interference from camera noise and dust. Contrast-limited adaptive histogram equalization (CLAHE) then divides the image into blocks to manipulate local contrast, avoiding overexposure caused by global equalization and enhancing product texture in shadowed areas or areas with strong light reflections. The cropping strength is dynamically adjusted based on the entropy of the noise-treated image. Higher entropy indicates rich image information (such as in densely packed shelves), and the cropping range is reduced to preserve more detail. Lower entropy indicates redundant information (such as in open aisles), and the cropping range is increased to eliminate insignificant background, allowing for precise focus on product details. In warehouse scenarios, the entropy of densely packed shelves is typically between 6.5 and 8 (the theoretical maximum entropy of an 8-bit image is 8). Open aisles, dominated by a single floor / wall surface with concentrated grayscale, tend to have an entropy between 2 and 4. Therefore, when the entropy value is above 6.5, it is identified as an information-dense area, and the cropping range is reduced from the standard 50% to 20% to ensure that cargo details are not miscropped. If the value is below 6.5, the cropping range is increased to 70%-80% to eliminate redundant background. Wavelet noise reduction combined with soft shrinkage prevents noise from misleading subsequent recognition, while CLAHE improves image clarity and avoids blurring of cargo caused by uneven warehouse lighting. Adaptive cropping based on image entropy reduces computing power consumption on edge devices while ensuring the integrity of key features, providing accurate input for convolutional neural networks.

[0028] In some embodiments of the present application, when determining whether there are errors in the placement of goods in the corresponding monitoring area based on the target frame image and the convolutional neural network model, it includes: finding the model parameters of the convolutional neural network model based on grid search, establishing a convolutional neural network model, the convolutional neural network model includes a convolution layer, a pooling layer and a fully connected layer, and a binary classification output layer is set in the fully connected layer, obtaining a sample data set, and dividing the sample data set into a training set and a test set, training the convolutional neural network model according to the training set, and substituting the test set into the trained convolutional neural network model to determine the prediction accuracy. If the prediction accuracy of the currently trained convolutional neural network model is greater than or equal to the prediction accuracy of the previously trained convolutional neural network model, and the prediction accuracy of the currently trained convolutional neural network model is greater than 0.85, then the training is stopped; otherwise, different network layers are differentially adjusted according to L1 and L2 regularization, and training is continued, and the target frame image is substituted into the convolutional neural network model after the training is stopped to determine whether there are errors in the placement of goods in the corresponding monitoring area.

[0029] Specifically, grid search constructs a convolutional neural network model by exhaustively searching for the optimal parameter combination in the parameter space, such as the size of the convolution kernel and the pooling step size. This avoids the subjectivity and blindness of manual parameter adjustment, allowing the convolution layer to efficiently extract key features such as the edges and textures of the goods. The pooling layer effectively compresses the data volume and retains important information. The fully connected layer maps the features to "error" (1) or "normal" (0) judgment results through a binary classification output layer (with a Sigmoid activation function). This improves the initial performance of the model by 15%-20% and improves the accuracy and stability of the model. The sample dataset contains various images of the correct placement of goods in each monitoring area under different lighting conditions (the image angle is consistent with the image angle of the current frame image). The sample dataset is divided into a training set and a test set, usually with a ratio of 4:1 to ensure the generalization ability of the model. The training set is used to train the model and iteratively update the model parameters, and the test set is used to evaluate the performance of the trained model. The prediction accuracy reflects the performance of the model on unknown data and is an important indicator for evaluating model performance. During training, if the current prediction accuracy is ≥ the previous prediction accuracy, and the prediction accuracy is >85%, training is terminated and the model is considered to have reached a stable performance level. Otherwise, the model's performance is considered to have not reached a stable performance level, and differential adjustments are made through L1 and L2 regularization. Specifically, L2 regularization is applied to the convolutional layers to limit the sum of squared weights and prevent excessive feature extraction and thus fitting noise. L1 regularization is applied to the fully connected layers to sparse the weights, remove redundant features, and ensure the lightweight model. This layered regularization adjustment ensures the model's prediction accuracy in complex warehouse environments (such as sudden changes in lighting and obstructions caused by goods). Finally, the target frame is input into the trained convolutional neural network model, which outputs a judgment result on whether there are any incorrectly placed goods in the corresponding monitoring area.

[0030] In some embodiments of the present application, when there is an error in the placement of goods, a corresponding warning is issued according to the regional priority of the corresponding monitoring area, including: when the corresponding monitoring area is a first-level priority, a red warning is triggered, when the corresponding monitoring area is a second-level priority, an orange warning is triggered, and when the corresponding monitoring area is a third-level priority, a yellow warning is triggered. When the red warning / orange warning / yellow warning is triggered, the time of triggering the warning is positively correlated with the number of errors in the placement of goods.

[0031] Specifically, the first-level priority monitoring areas (such as areas for storing dangerous goods and high-value goods) trigger red alerts, because incorrect placement of their goods may cause safety accidents or major economic losses. As the highest level of warning, the red alert can urgently remind relevant personnel in the first time to ensure emergency response, while the second-level priority monitoring areas (such as storage areas for conventional best-selling products) trigger orange alerts. Although errors in such areas are not as serious as those in the first-level priority, they will also affect the efficiency of goods outbound delivery, and therefore trigger orange alerts, which are one level lower than the red alert. The third-level priority monitoring areas (such as slow-moving goods and spare material areas) trigger yellow alerts. Their errors have less impact on the overall warehouse operation. The yellow alert can avoid occupying too many management resources, and the corresponding warning time is positively correlated with the number of incorrectly placed goods, thereby conveying the severity of the warehouse goods problem through the length of time. For example: when only individual goods are misplaced in a certain monitoring area, the warning time is short to avoid excessive interference with the warehouse monitoring work. If batches of goods are mixed or misplaced, the warning time is extended to emphasize the concentration of the problem. Batch errors usually reflect process loopholes or inadequate implementation of operating specifications, requiring relevant personnel to invest more time to investigate the root cause. By using graded warnings to distinguish the degree of urgency, the efficiency of resource scheduling is ensured. In addition, by conveying the scale of the problem through differences in duration, the overall management cost is reduced, which not only avoids the internal consumption of resources caused by indiscriminate warnings, but also prevents problems from being missed due to insufficient warnings, thereby improving the reliability of warehouse monitoring.

[0032] In some embodiments of the present application, when there is no error in the placement of goods, an edge processing strategy or a cloud processing strategy is determined based on the corresponding regional priority, including: determining the real-time anomaly detection tasks of the first-level priority, the second-level priority, and the third-level priority as the edge processing strategy, and the edge processing strategy is executed by the edge device including the detection of over-limit cargo stacking and the detection of illegal entry of personnel, and determining the trend analysis tasks and cargo traceability tasks of the first-level priority, the second-level priority, and the third-level priority as the cloud processing strategy, and the cloud processing strategy is executed by the cloud server including inventory change trend analysis and historical backtracking of abnormal events.

[0033] Specifically, when there is no error in the placement of goods, the processing strategy is allocated according to the regional priority, so as to divide the work according to the adaptability of the task characteristics and the processing scenario. Real-time anomaly detection tasks (over-stacked goods, illegal entry of personnel) have high requirements for response speed. Over-stacked goods may instantly cause the collapse of goods, and illegal entry of personnel may cause safety accidents or loss of goods. Regardless of the level of regional priority, they are all executed by edge devices. Edge devices are relatively close to the monitoring area and can directly obtain video streams and perform calculations quickly to avoid the risk of delayed intervention due to delays in data transmission to the cloud. Trend analysis tasks (inventory change trends) and cargo traceability tasks (historical backtracking of abnormal events) are uniformly handled by the cloud server. Since trend analysis requires the integration of long-term The data from multiple monitoring areas across the country can be used to discover inventory fluctuations. This relies on the accumulation of a large amount of historical records and in-depth calculations. The traceability task requires tracing back the complete process of a specific event, involving the retrieval of monitoring video streams from multiple time periods. It has high requirements for storage capacity and computing power. Cloud servers have the ability to centrally store massive amounts of data and can mobilize computing resources to complete the calculation of complex models. The edge processing strategy or cloud processing strategy is determined by regional priority, ensuring the rapid identification of security risks and violations, and avoiding the spread of risks. The processing strategies of cloud servers and edge devices are separated, which not only avoids delays caused by insufficient computing power of edge devices, but also prevents waste of resources caused by processing real-time tasks in the cloud, thus forming a collaborative operation system and ensuring the reliability of resource scheduling and utilization. In some embodiments of the present application, when determining the load data volume based on the edge processing strategy or the cloud processing strategy, it includes: the load data volume of the edge processing strategy includes the current frame image extraction data volume and the number of concurrent tasks, and the load data volume of the cloud processing strategy includes the cloud transmission data volume and the new task data transmission data volume.

[0034] In some embodiments of the present application, when migrating edge processing tasks in an edge processing strategy to a cloud server based on the load data volume, it includes: recording the load data volume of the edge processing strategy as the edge load data volume, comparing the edge load data volume with the edge load data volume threshold, the edge load data volume threshold is determined based on the rated computing power of the edge device, and when the edge load data volume is greater than or equal to the edge load data volume threshold, the task migration priority is sorted, and the priority sorting is determined based on the task real-time performance, data encryption level and migration bandwidth consumption. The real-time coefficient and data encryption level are positively correlated with the cargo storage requirements. The edge processing tasks are migrated in order from high to low based on the results of the task migration priority sorting, and the migration process adopts incremental data transmission.

[0035] Specifically, the division of load data volume reflects the precise adaptation of the task characteristics of edge devices and cloud servers. The edge processing strategy focuses on real-time operations. The amount of data extracted from the current frame image directly reflects the effective information scale of the single frame image after pre-processing. The number of concurrent tasks reflects the number of detection tasks processed simultaneously by the edge device (such as stacking detection of multiple shelves at the same time). The two together constitute the immediate computing power requirements of the edge device. The cloud processing strategy focuses on data-intensive tasks. The amount of data transmitted in the cloud covers the analysis results and basic features uploaded by the edge device. The amount of data transmitted for new tasks is for the historical data required for newly initiated trend analysis, traceability and other tasks. The two reflect the scale of input data during cloud processing. The edge load data volume threshold is determined based on the rated computing power of the device to ensure that it does not exceed the upper limit of the hardware load and avoid delays or crashes caused by overload of the edge device. When the edge load data volume is greater than or equal to the edge load data volume threshold, it is migrated according to task priority. Task migration priority is to balance processing efficiency and data security through multi-dimensional indicators. Task real-time is reflected in the sensitivity to processing delay. For example, the detection of over-limit cargo stacking in the first-level priority monitoring area requires immediate response (the storage requirements of first-level priority cargo are higher). The real-time coefficient is determined by hierarchical mapping and integration of multi-dimensional factors. First, for cargo storage requirements, they are divided into several levels according to their severity (such as safety risk level, environmental sensitivity, turnover rate, etc.). Each level corresponds to the same basic coefficient value. For example: cargo only requires one level of safety risk level, and cargo requires three levels of safety risk level, environmental sensitivity, and turnover rate. Whenever there is a level of cargo, the corresponding basic coefficient value accumulates higher. Data encryption levels are also determined based on the monitoring area's regional priority. Priority-level data, such as data from valuable electronic products, requires stronger encryption protection, and the data encryption level aligns with the regional priority of the monitoring area. Migration bandwidth consumption is determined by the network resource usage of the task data volume. Task migration priority is first sorted by real-time performance, then fine-tuned by data encryption level (tasks with higher encryption levels can be prioritized), and finally optimized by bandwidth consumption (tasks with lower bandwidth requirements are prioritized for the same priority). Edge processing tasks are migrated sequentially based on the task migration priority ranking, ensuring that high-urgency and high-sensitivity tasks receive priority access to cloud resources while avoiding migration congestion caused by excessive bandwidth usage. The migration process utilizes incremental data transmission, sending only the edge device's processed feature data and newly added changes (such as differences in cargo location) to the cloud. This reduces the transmission of duplicate data and accurately quantifies the data payload, providing an objective basis for task migration, avoiding resource waste caused by blind migration and ensuring efficient resource scheduling and utilization.

[0036] In some embodiments of the present application, when adjusting the resource allocation of the cloud server based on the migration complexity, it includes: the migration complexity is determined according to the data transmission delay and the conflict rate of similar tasks on the cloud, the conflict rate of similar tasks on the cloud is the overlapping ratio of the computing power resources required for the current migration edge processing task and the allocated resources on the cloud, and the resource allocation weight of the regional priority in the cloud server is adjusted based on the migration complexity.

[0037] Specifically, migration complexity is determined by combining two dimensions: data transmission latency and the conflict rate of similar tasks on the cloud. Data transmission latency reflects the efficiency of data exchange between edge devices and the cloud. It is affected by network stability and the type of data being transmitted. Higher latency means longer processing time after task migration, resulting in increased complexity. The conflict rate of similar tasks on the cloud focuses on resource competition. When the computing resources required by the migration task overlap significantly with the resource requirements of already running tasks on the cloud, the conflict rate increases, leading to task queuing or resource preemption, further increasing processing complexity. Each data transmission latency and conflict rate on the cloud corresponds to a baseline set value. Migration complexity is determined by taking the weighted sum of the two. When adjusting resource allocation weights based on migration complexity, the resource allocation weights of regional priorities on the cloud server are adjusted. Specifically, under normal circumstances, the resource allocation weights of priority levels 1, 2, and 3 on the cloud are 50%, 30%, and 20%, respectively. When the migration complexity of a task in a priority level monitoring area exceeds the migration complexity threshold set for priority level 1, its weight is increased to 60%, while that of priority levels 2 and 3 is reduced to 25% and 15%, respectively, to ensure that high-priority tasks quickly obtain sufficient resources. When the migration complexity of a task in a monitoring area with a second-level priority exceeds the migration complexity threshold for that priority, the resource allocation weight for the first-level priority remains unchanged at 50%, the resource allocation weight for the second-level priority increases to 40% to prioritize conflict resolution, and the resource allocation weight for the third-level priority decreases to 10%. When the migration complexity of a task in a monitoring area with a third-level priority exceeds the migration complexity threshold for that priority (e.g., for a temporary inventory requiring historical data tracing), the resource allocation weight for the third-level priority increases to 25%, leaving the resource allocation weights for the first- and second-level priorities at 50% and 25%, respectively. This ensures the progress of essential tasks without impacting monitoring requirements. The resource allocation weights for the first, second, and third-level priorities in the cloud can be adjusted up or down based on migration complexity. This dynamic adjustment mechanism preserves the core priority ranking while enabling flexible allocation based on migration complexity. This avoids waste or congestion caused by resource rigidification, reduces processing delays due to resource mismatches, and prevents excessive resource usage by low-priority tasks, thus achieving a balance between efficiency and stability in resource scheduling within cloud-edge collaboration. In summary, the beneficial effects of the present invention are as follows: by dividing the regional priorities of the monitoring areas, differentiated and precise allocation of resources is achieved, the risk of monitoring delays or missed detections due to resource preemption in key areas is avoided, and the pertinence and reliability of warehouse monitoring are improved. According to the different processing strategies of edge devices and cloud servers, the requirements of real-time performance and computing power are effectively balanced, and the limitations of a single processing mode are made up. The task migration mechanism based on the load data volume can dynamically match processing requirements and resource capabilities. It avoids overloading of edge devices or idle cloud resources, thereby improving overall resource utilization and ensuring the efficient operation of video analysis tasks. The resource allocation of cloud servers is dynamically adjusted in combination with the complexity of migration, thereby optimizing resource configuration in a targeted manner, adapting more resources to relatively complex migration tasks to ensure processing efficiency; and streamlining resources for simple tasks to reduce operating costs, thereby achieving a balance between efficiency and stability of resource scheduling.

[0038] In another preferred embodiment based on the above embodiment, refer to Figure 2 As shown, this embodiment provides a resource scheduling system based on cloud-edge collaborative warehouse video analysis, which is used to apply the above-mentioned resource scheduling method based on cloud-edge collaborative warehouse video analysis, including: The acquisition and analysis unit is configured to obtain the surveillance video streams of each camera in the warehouse and determine the monitoring area of ​​each camera, and divide all the monitoring areas into regional priorities according to the storage requirements of the goods; a judgment processing unit configured to extract the surveillance video stream of each surveillance area to determine a current frame image, preprocess the current frame image to determine a target frame image, and determine whether there is a cargo placement error in the corresponding surveillance area based on the target frame image and a convolutional neural network model; a strategy determination unit configured to, when a cargo placement error occurs, issue a corresponding warning based on the regional priority of the corresponding monitoring area, and, when no cargo placement error occurs, determine an edge processing strategy or a cloud processing strategy based on the corresponding regional priority; The resource scheduling unit is configured to determine the load data volume based on the edge processing strategy or the cloud processing strategy, migrate the edge processing tasks in the edge processing strategy to the cloud server according to the load data volume, and adjust the resource allocation of the cloud server based on the migration complexity.

[0039] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0040] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0041] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0042] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0043] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A resource scheduling method based on warehouse video analysis in cloud-edge collaboration, characterized in that: include: Obtain surveillance video streams from all cameras in the warehouse and determine the monitoring area of ​​each camera, and prioritize all monitoring areas based on cargo storage requirements; Extracting the surveillance video stream of each monitoring area to determine the current frame image, preprocessing the current frame image to determine the target frame image, and determining whether there is any cargo placement error in the corresponding monitoring area based on the target frame image and the convolutional neural network model; When there is a cargo placement error, a corresponding warning is issued based on the regional priority of the corresponding monitoring area. When there is no cargo placement error, the edge processing strategy or cloud processing strategy is determined based on the corresponding regional priority. The load data volume is determined based on the edge processing strategy or the cloud processing strategy, and the edge processing tasks in the edge processing strategy are migrated to the cloud server according to the load data volume, and the resource allocation of the cloud server is adjusted based on the migration complexity.

2. The resource scheduling method for warehouse video analysis based on cloud-edge collaboration according to claim 1 is characterized in that: When all monitoring areas are divided into regional priorities according to cargo storage requirements, they include: determining a complexity score based on cargo storage requirements, wherein the complexity score is positively correlated with the cargo storage requirements; determining a first complexity score, a second complexity score, and a third complexity score, wherein the first complexity score is greater than the second complexity score, and the second complexity score is greater than the third complexity score; When the complexity score is greater than or equal to the first complexity score, the corresponding monitoring area is determined as a first-level priority; When the complexity score is less than the first complexity score and greater than the second complexity score, the corresponding monitoring area is determined as a second-level priority; When the complexity score is less than or equal to the second complexity score, the corresponding monitoring area is determined to be a third level priority; The resource inputs for the first-level priority, the second-level priority, and the third-level priority decrease in sequence.

3. The resource scheduling method based on cloud-edge collaboration warehouse video analysis according to claim 2 is characterized in that: When the current frame image is preprocessed to determine the target frame image, the method includes: The current frame image is subjected to wavelet decomposition and noise processing based on BayesShrink soft shrinkage, and the noise processing result is subjected to limited contrast adaptive histogram equalization, and the cropping intensity is adjusted according to the image entropy after noise processing to determine the target frame image.

4. The resource scheduling method for warehouse video analysis based on cloud-edge collaboration according to claim 3 is characterized in that: When determining whether there is a cargo placement error in the corresponding monitoring area based on the target frame image and the convolutional neural network model, it includes: Based on grid search, model parameters of a convolutional neural network model are found, and a convolutional neural network model is established. The convolutional neural network model includes a convolution layer, a pooling layer, and a fully connected layer, and a binary classification output layer is provided in the fully connected layer; Obtaining a sample data set and dividing the sample data set into a training set and a test set, training the convolutional neural network model based on the training set, and substituting the test set into the trained convolutional neural network model to determine the prediction accuracy; If the prediction accuracy of the currently trained convolutional neural network model is greater than or equal to the prediction accuracy of the previously trained convolutional neural network model, and the prediction accuracy of the currently trained convolutional neural network model is greater than 0.85, then training is stopped. Otherwise, different network layers are differentially adjusted according to L1 and L2 regularization, and training is continued. The target frame image is substituted into the convolutional neural network model after stopping training to determine whether there is any error in the placement of goods in the corresponding monitoring area.

5. The resource scheduling method for warehouse video analysis based on cloud-edge collaboration according to claim 4 is characterized in that: When there is a mistake in the placement of goods, corresponding warnings will be issued according to the regional priority of the corresponding monitoring area, including: When the corresponding monitoring area is at the first-level priority level, a red alert is triggered; When the corresponding monitoring area is at the second-level priority level, an orange warning is triggered; When the corresponding monitoring area is at the third level of priority, a yellow warning is triggered; When a red warning / orange warning / yellow warning is triggered, the time it takes to trigger the warning is positively correlated with the number of incorrectly placed goods.

6. The resource scheduling method for warehouse video analysis based on cloud-edge collaboration according to claim 5 is characterized in that: When there are no errors in the placement of goods, the edge processing strategy or cloud processing strategy is determined based on the corresponding regional priorities, including: Determine the first-level priority, second-level priority, and third-level priority real-time anomaly detection tasks as the edge processing strategy, wherein the edge processing strategy includes cargo stacking overrun detection and personnel illegal entry detection for edge devices; The trend analysis tasks and cargo tracing tasks of the first-level priority, the second-level priority and the third-level priority are determined as the cloud processing strategy, and the cloud processing strategy is executed by the cloud server including inventory change trend analysis and abnormal event history backtracking.

7. The resource scheduling method for warehouse video analysis based on cloud-edge collaboration according to claim 6 is characterized in that: When determining the load data amount based on the edge processing strategy or the cloud processing strategy, it includes: The load data volume of the edge processing strategy includes the current frame image extraction data volume and the number of concurrent tasks; The load data volume of the cloud processing strategy includes the cloud transmission data volume and the new task data transmission data volume.

8. The resource scheduling method for warehouse video analysis based on cloud-edge collaboration according to claim 7 is characterized in that: When migrating the edge processing task in the edge processing strategy to the cloud server according to the load data volume, it includes: Recording the load data volume of the edge processing strategy as edge load data volume, and comparing the edge load data volume with an edge load data volume threshold, where the edge load data volume threshold is determined according to the rated computing power of the edge device; When the edge load data volume is greater than or equal to the edge load data volume threshold, the tasks are prioritized for migration. The priority is determined based on the real-time performance of the tasks, the data encryption level, and the migration bandwidth consumption. The real-time coefficient and the data encryption level are positively correlated with the cargo storage requirements. Based on the results of task migration priority sorting, edge processing tasks are migrated from high to low, and the migration process adopts incremental data transmission.

9. The resource scheduling method for warehouse video analysis based on cloud-edge collaboration according to claim 8 is characterized in that: When adjusting resource allocation of the cloud server based on migration complexity, it includes: The migration complexity is determined based on the data transmission delay and the conflict rate of similar tasks on the cloud. The conflict rate of similar tasks on the cloud is the overlap ratio between the computing power resources required for the current migrating edge processing task and the resources allocated on the cloud. The resource allocation weight of the regional priority in the cloud server is adjusted based on the migration complexity.

10. A resource scheduling system based on cloud-edge collaborative warehouse video analysis, used to apply the resource scheduling method based on cloud-edge collaborative warehouse video analysis according to any one of claims 1 to 9, characterized in that: include: The acquisition and analysis unit is configured to obtain the surveillance video streams of each camera in the warehouse and determine the monitoring area of ​​each camera, and divide all the monitoring areas into regional priorities according to the storage requirements of the goods; a judgment processing unit configured to extract the surveillance video stream of each surveillance area to determine a current frame image, preprocess the current frame image to determine a target frame image, and determine whether there is a cargo placement error in the corresponding surveillance area based on the target frame image and a convolutional neural network model; a strategy determination unit configured to, when a cargo placement error occurs, issue a corresponding warning based on the regional priority of the corresponding monitoring area, and, when no cargo placement error occurs, determine an edge processing strategy or a cloud processing strategy based on the corresponding regional priority; The resource scheduling unit is configured to determine the load data volume based on the edge processing strategy or the cloud processing strategy, migrate the edge processing tasks in the edge processing strategy to the cloud server according to the load data volume, and adjust the resource allocation of the cloud server based on the migration complexity.

Images