An image processing method and electronic equipment based on end-cloud cooperation

Abstract

Embodiments of the present application disclose an image processing method and electronic equipment based on end-cloud cooperation. The method comprises: determining a target task quantity corresponding to a transmission channel based on an original task quantity and / or an allocated task quantity corresponding to the transmission channel; determining a task quantity proportion of the target task quantity relative to a target task quantity sum corresponding to each transmission channel, selecting a target channel based on the task quantity proportion; allocating the original task quantity corresponding to the target channel to all transmission channels until a calculation stop condition is met; and distributing intermediate data processed by a mapping node to a reduction node according to a composition feature of the target task quantity corresponding to each transmission channel, wherein the intermediate data is transmitted by each transmission channel. The above scheme can efficiently realize balanced distribution of data in the case of cooperation between the front end and the back end, and only performs balanced calculation on the original task quantity of the same target channel, thereby avoiding re-distribution after the original task quantity is distributed and increasing the analysis difficulty of the back end.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to an image processing method based on end-cloud cooperation and an electronic device. BACKGROUND

[0002] With the rapid development of security monitoring, the data volume is growing explosively, and large-scale data contains infinite value. Meanwhile, big data processing brings new challenges. Focusing on data processing in a big data environment, the parallel computing model MapReduce of Hadoop processes large-scale complex data in the fastest way through the idea of decomposition and combination.

[0003] Massive big data is processed in large-scale parallel through the MapReduce computing framework, which mainly includes two stages of Map and Reduce. The Map stage calculates according to HDFS "sharding" as the minimum calculation unit, and the processed data in the memory is first written into a logical storage space, a ring buffer. If the data in the ring buffer reaches a preset threshold (0.8 by default), the results of multiple Maps are subjected to an initial aggregation operation. The combined results are mapped to the predetermined partition through a hash function. Due to hash collision, multiple same hash results are mapped to the same partition. As more and more data are mapped to a certain partition, data skew is formed.

[0004] In a city monitoring system, the prosperity of each region and street is different, which leads to a very obvious difference in the target data volume collected by cameras in different sections. The image data transmitted in the same transmission channel of the camera is allocated to a Reduce node for processing. Since the image data transmitted in different camera transmission channels needs to be analyzed differently, the computing pressure of different Reduce nodes is different, so data skew is generated in this scenario. SUMMARY

[0005] The embodiment of the present application provides an image processing method based on end-cloud cooperation and an electronic device, so as to solve the problem of data skew of image data in the MapReduce computing framework under the condition of end-cloud cooperation.

[0006] According to an aspect of the present application, an image processing method based on end-cloud cooperation is provided, which comprises:

[0007] determining, for each transmission channel, a target task amount corresponding to the transmission channel based on an original task amount and / or an allocated task amount corresponding to the transmission channel, wherein the original task amount is received from a front-end image collector and reflects a task amount corresponding to analysis of image data transmitted through the transmission channel in a first image processing period, and the allocated task amount is a task amount allocated to the transmission channel in a previous calculation round;

[0008] selecting, for each transmission channel, a target channel based on a task amount proportion of the target task amount relative to a sum of target task amounts corresponding to each transmission channel;

[0009] allocating the original task amount corresponding to the target channel to all transmission channels and entering a next calculation round until a calculation stop condition is met;

[0010] in a second image processing period, distributing, according to a composition feature of the target task amount corresponding to each transmission channel, intermediate data processed by a mapping node to a reduction node, wherein the first image processing period is before the second image processing period.

[0011] According to an aspect of the present application, an image processing device based on end-to-cloud collaboration is provided, and the device comprises:

[0012] a target task amount determination module configured to determine, for each transmission channel, a target task amount corresponding to the transmission channel based on an original task amount and / or an allocated task amount corresponding to the transmission channel in a target calculation round, wherein the original task amount is received from a front-end image collector and reflects a task amount corresponding to analysis of image data transmitted through the transmission channel in a first image processing period, and the allocated task amount is a task amount allocated to the transmission channel in a previous calculation round;

[0013] a target channel selection module configured to select, for each transmission channel, a target channel based on a task amount proportion of the target task amount relative to a sum of target task amounts corresponding to each transmission channel;

[0014] a calculation module configured to allocate the original task amount corresponding to the target channel to all transmission channels and enter a next calculation round until a calculation stop condition is met;

[0015] a distribution module configured to distribute, in a second image processing period, according to a composition feature of the target task amount corresponding to each transmission channel, intermediate data processed by a mapping node to a reduction node, wherein the first image processing period is before the second image processing period.

[0016] According to another aspect of the present application, an electronic device is provided, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory connected with the at least one processor in communication; wherein,

[0019] the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image processing method based on end-cloud collaboration of any of the embodiments of the present application.

[0020] According to another aspect of the present application, a computer readable storage medium is provided, the computer readable storage medium storing computer instructions for causing a processor to implement the image processing method based on end-cloud collaboration of any of the embodiments of the present application when executed by the processor.

[0021] According to another aspect of the present application, a computer program product is provided, the computer program product comprising a computer program which, when executed by a processor, implements the image processing method based on end-cloud collaboration of any of the embodiments of the present application.

[0022] The technical scheme of the embodiment of the present application is that, in the target computing round, for each transmission channel, the target task quantity corresponding to the transmission channel is determined based on the original task quantity and / or the allocated task quantity corresponding to the transmission channel; the original task quantity is received from a front-end image collector and reflects the task quantity corresponding to the analysis of the image data transmitted through the transmission channel in the first image processing period; the allocated task quantity is the task quantity allocated to the transmission channel in the last computing round; thus, the target task quantity corresponding to the transmission channel in the target computing round is accurately calculated and updated, so as to further guide the calculation strategy in the target computing round. For each transmission channel, the task quantity proportion of the target task quantity relative to the sum of the target task quantities corresponding to each transmission channel is determined, and the target channel is selected from each transmission channel based on the task quantity proportion, so that the target channel with greater data tilt influence is targeted to take the balancing strategy, so as to improve the balancing efficiency. The original task quantity corresponding to the target channel is allocated to all transmission channels and enters the next computing round until the calculation stop condition is met, the balanced allocation of the task quantity is realized, the situation of data tilt generated in the stage allocated to the reduction node is solved, and only one balanced allocation is performed on the original task quantity of the same target channel, so that the back-end only needs to perform one analysis when analyzing the image data of the same transmission channel, thereby avoiding the problem that the original task quantity corresponding to the same target channel is allocated multiple times and the back-end needs to perform multiple rounds of analysis. In the second image processing period, according to the composition characteristics of the target task quantities corresponding to each transmission channel, the intermediate data processed by the mapping node after the image data transmitted through each transmission channel is allocated to the reduction node, so that the balanced allocation of each computing round is implemented at the reduction node, load balancing is realized at the reduction node, and the problem of data tilt is solved.

[0023] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0025] Figure 1 A flowchart of an image processing method based on end-cloud cooperation provided by the embodiment of the present application;

[0026] Figure 2 A MapReduce framework principle diagram provided by the embodiment of the present application;

[0027] Figure 3 A flow chart of an image processing method based on end-cloud cooperation provided for another embodiment of the application is shown in FIG. 6;

[0028] Figure 4 A flow chart of an image processing method based on end-cloud cooperation provided for another embodiment of the application is shown in FIG. 6;

[0029] Figure 5 A flow chart of an image processing method based on end-cloud cooperation provided for another embodiment of the application is shown in FIG. 6;

[0030] Figure 6 A fisheye camera screen schematic diagram provided for another embodiment of the application is shown in FIG. 6;

[0031] Figure 7 A four transmission channel screen schematic diagram provided for another embodiment of the application is shown in FIG. 6;

[0032] Figure 8 A scanning distribution schematic diagram provided for another embodiment of the application is shown in FIG. 6;

[0033] Figure 9 A structure schematic diagram of an image processing device based on end-cloud cooperation provided for an embodiment of the application is shown in FIG. 6;

[0034] Figure 10 A structure schematic diagram of an electronic device provided for an embodiment of the application is shown in FIG. 6. DETAILED DESCRIPTION

[0035] In order to make the personnel in the art better understand the scheme of the application, the technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only a part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by the person of ordinary skill in the art without creative labor should belong to the scope of protection of the application.

[0036] It should be noted that the terms "first", "second", "third", "fourth", "actual", "preset" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0037] Figure 1 A flowchart of an image processing method based on end-cloud cooperation is provided for the embodiments of the present application. The embodiments of the present application can be applicable to equalize distribution of image data to avoid the situation of unbalanced load of reduction nodes. The method can be executed by an image processing device based on end-cloud cooperation, which can be realized in the form of hardware and / or software, and can be configured in an electronic device. As shown in the figure, the method comprises: Figure 1

[0038] S110, in the target computing round, for each transmission channel, determining the target task quantity corresponding to the transmission channel based on the original task quantity and / or the allocated task quantity corresponding to the transmission channel; wherein the original task quantity is received from the front-end image collector, reflecting the task quantity corresponding to the analysis of the image data transmitted through the transmission channel in the first image processing period; the allocated task quantity is the task quantity allocated to the transmission channel in the last computing round.

[0039] Wherein, the embodiments of the present application are applicable to the situation of data skew when processing image data in the MapReduce computing framework, which includes mapping nodes and reduction nodes. The target computing round refers to the computing round corresponding to the scheme described in the embodiments of the present application, which is any computing round before the end of the computation. After the end of the target computing round, the next computing round is entered, then the target computing round is the last computing round, and the next computing round is the target computing round. The transmission channel is a channel for transmitting image data from the front-end image collector to the back-end server. One front-end image collector may correspond to one transmission channel, or may correspond to at least two transmission channels, which is determined according to the specific situation. For example, for basic shooting or single-camera system, one transmission channel is generally configured. Multi-channel image collector may transmit image data through one transmission channel, or may transmit image data through multiple transmission channels under specific business requirements.​

[0040] The original task amount is the amount of tasks required for analyzing the image data collected by the front-end image collector and directly transmitted via the transmission channel. The limitation of the image data is the image data transmitted via the transmission channel in the first image processing period, i.e., historical data. In the case where the scheme in the embodiment of the present application is executed by the mapping node, the original task amount is sent by the front-end image collector to the mapping node. The allocated task amount is the amount of tasks allocated to the transmission channel in the last calculation round, i.e., the amount of tasks allocated to the transmission channel when the original task amounts of other transmission channels are reallocated in the last calculation round. The task amount for analyzing the image data can be obtained by counting the image data collected by the front-end image collector in the first image processing period. The counting process can be specifically as follows: identifying the image, determining the number of targets of each type contained in the image, and calculating the task amount for analyzing the target of each type according to the number of the target of each type and the weight corresponding to the analysis of the target of each type. The weight reflects the computing resources required by the back-end server, i.e., the MapReduce computing framework, for analyzing the target of each type, and after quantization, the weight can be represented by the task amount for analyzing the target of each type. The task amount is a relative value relative to the task amount for analyzing other types of targets, and is not an absolute value. For the entire image, the sum of the task amounts corresponding to the targets of various types contained in the image is taken as the original task amount corresponding to the image data. For example, in the first image processing period, M vehicles, B non-motor vehicles, and P pedestrians are identified in the collected image. The back-end server running the MapReduce assigns different weights to each business according to the computing resources consumed by each intelligent analysis task, for example: the structured analysis task of the pedestrian is the heaviest, with a weight of 8; the structured analysis task of the vehicle is relatively heavy, with a weight of 5; the structured analysis task of the non-motor vehicle is light, with a weight of 3; and the person number counting task is the lightest, with a weight of 1. In the first image processing period, the task amount can be counted as . wherein i represents the target serial number (motor vehicle, non-motor vehicle, and pedestrian, respectively), n represents the total number of targets, and task i represents the weight of the analysis task corresponding to the target.

[0041] In the embodiment of the present application, for each transmission channel corresponding to each front-end image collector, the target task quantity corresponding to the transmission channel is determined based on the original task quantity and / or the allocated task quantity corresponding to the transmission channel in the target computing round. If the original task quantity of the transmission channel is redistributed in the last computing round, since the original task quantity of the transmission channel is all allocated in the last computing round, the target task quantity of the transmission channel in the target computing round only has the allocated task quantity, and no longer has the original task quantity. If the original task quantity of the transmission channel is not allocated in the last computing round, or the target computing round is the first computing round, and there is no last computing round, the target task quantity of the transmission channel only includes the original task quantity. If the target computing round is not the first computing round, and the original task quantity of the transmission channel is allocated to other transmission channels in the last computing round, the transmission channel corresponds to the original task quantity and the allocated task quantity, and the target task quantity is determined according to the original task quantity and the allocated task quantity. Specifically, the sum of the original task quantity and the allocated task quantity can be taken as the target task quantity, reflecting the total task quantity allocated to the transmission channel. The scheme in the embodiment of the present application is generally executed after the first image processing period ends, and can be considered to enter the second image processing period from S110.

[0042] S120, for each transmission channel, determining a task quantity proportion of the target task quantity relative to a target task quantity sum corresponding to each transmission channel, and selecting a target channel from each transmission channel based on the task quantity proportion.

[0043] The target task quantity sum is the sum of the target task quantities corresponding to each transmission channel. For each transmission channel, the task quantity proportion of the target task quantity relative to the target task quantity sum corresponding to each transmission channel is determined. Since in the current MapReduce computing framework, the reduction node at the back end is one-to-one corresponding to the transmission channel, that is, the image data transmitted in the same transmission channel is finally transmitted to the same reduction node after being processed by the mapping node. In this case, the task quantity proportion corresponding to the transmission channel also reflects the calculation amount finally transmitted to the corresponding reduction node, and the calculation amount of the reduction node is positively correlated with the task quantity proportion corresponding to the transmission channel. The greater the task quantity proportion corresponding to the transmission channel, the greater the calculation amount of the corresponding reduction node, and the greater the calculation pressure. The greater the difference between the task quantity proportions corresponding to different transmission channels, the more serious the data skew generated at the reduction node end. Based on this, the target channel that causes the greatest data skew or affects the degree of data skew can be selected from the transmission channels according to the task quantity proportions corresponding to each transmission channel, and the target task quantity corresponding to the target channel is balanced to balance the calculation amount at the reduction node.

[0044] In the embodiments of the present application, in order to balance the calculation amount of the reduction node, the target channel can be selected from the transmission channels based on the task amount proportion, specifically, a transmission channel that meets at least one of the following conditions is selected as the target channel: the task amount proportion is the largest, the difference between the task amount proportions is the largest, and the task amount proportion exceeds a preset task amount proportion threshold. In this way, the target task amount in the target channel is allocated and calculated based on the subsequent scheme, so as to reduce the calculation pressure of the reduction node corresponding to the target channel.

[0045] In S130, the original task amount corresponding to the target channel is allocated to all transmission channels, and the next calculation round is entered until the calculation stop condition is met.

[0046] For example, the original task amount corresponding to the target channel can be allocated to all transmission channels, that is, all transmission channels share the original task amount corresponding to the target channel, so as to reduce the target task amount corresponding to the target channel, and further reduce the task amount allocated to the reduction node corresponding to the target channel, thereby realizing the balanced processing of the image data.

[0047] In the embodiments of the present application, the original task amount corresponding to the target channel can be allocated to all transmission channels, which can be uniform allocation or non-uniform allocation, as long as the target task amount corresponding to the target channel is reduced to a certain extent, so as to improve the data skew in the reduction node. In order to realize more effective balancing and balancing in the least calculation round, the original task amount corresponding to the target channel can be evenly allocated to each transmission channel. In this case, all transmission channels also include the target channel.

[0048] In the embodiments of the present application, after the target calculation round allocation is completed, the next calculation round can be entered, that is, the steps of S110-S130 are re-executed to allocate the original task amount in the target channel to realize data balancing and solve the data skew. If the calculation stop condition is met, the calculation is stopped, and the target task amount corresponding to each transmission channel is updated, that is, the target task amount of each transmission channel is updated according to the allocation result of the original task amount of the target channel after the last calculation round. The calculation stop condition can be determined according to the analysis of the task amount proportion corresponding to each transmission channel, for example, the task amount proportion of each transmission channel is less than a certain threshold, or the difference between the task amount proportions of the transmission channels is less than a certain threshold. It can also be determined according to the calculation round, for example, the calculation round reaches a preset round.

[0049] S140, in the second image processing cycle, according to the composition characteristics of the target task amount corresponding to each transmission channel, the intermediate data of the image data transmitted through each transmission channel and processed by the mapping node is distributed to the reduction node; wherein the first image processing cycle is located before the second image processing cycle.

[0050] For example, after the end of the above calculation round, the target task amount corresponding to each transmission channel can be updated, and the composition characteristics of the target task amount can be determined according to the original task amount and the allocated task amount constituting the target task amount. For example, the target task amount corresponding to a transmission channel channel1 includes original task amount 4 + allocated task amount 3 = target task amount 7. According to the composition of the target task amount, the composition characteristics of the target task amount can be determined, that is, which task amount is the original task amount, which task amount is the allocated task amount, and which target channel the allocated task amount corresponds to the original task amount allocation. According to the composition characteristics of the target task amount corresponding to each transmission channel, it can be determined that for the image data transmitted through each transmission channel, which data needs to be directly allocated to the corresponding reduction node after being processed by the mapping node, and which data needs to be evenly allocated.

[0051] Specifically, in the second image processing cycle, the image data transmitted through each transmission channel is transmitted to the cloud distributed cluster, the cloud distributed cluster stores the image data, and the image data is divided into multiple data blocks, and the size of each data block is aMb by default. Each data block is allocated to a mapping node for processing, the mapping node reads the input data block and executes the Map function locally, the Map function processes each record of the input data block and outputs the key-value pair Key-Value. In the security business, the uniquely configured transmission channel ID number can be used as the Key value, and the Value value is the binary data or feature data of the image data. If the total number of partitions is n, which corresponds to n reduction nodes one by one, first calculate the hash value of Key according to the hash function, and then distribute it to the corresponding partition. For example, Figure 2As shown, in the case of unknown data skew, the mapping node generates intermediate data, that is, key-value pairs, after computing the image data, and the intermediate data is allocated to the partition corresponding to the transmission channel under the partition function by default, and the reduction node downloads the intermediate data from the corresponding partition through the network for computing. In the embodiment of the present application, in order to solve the data skew situation, the intermediate data is not directly allocated to the reduction node corresponding to the transmission channel in the default case, but according to the composition characteristics of the target task amount corresponding to the transmission channel, the intermediate data after the image data corresponding to the original task amount in the target task amount is processed by the mapping node is directly allocated to the reduction node corresponding to the transmission channel, and the image data corresponding to the allocation task amount in the target task amount is actually transmitted from the target channel, and the mapping node needs to allocate the intermediate data after processing the image data corresponding to the allocation task amount of the transmission channel transmitted by the target channel to the reduction node corresponding to the transmission channel, so as to reduce the computing pressure of the reduction node corresponding to the target channel, realize the equalization of data, and alleviate the data skew situation. In the process of allocation, the intermediate data after processing the image data corresponding to the allocation task amount can increase a new identifier without losing the original transmission channel identifier to realize reallocation in each reduction node.

[0052] The technical scheme of the embodiment of the present application is that, in the target computing round, for each transmission channel, the target task quantity corresponding to the transmission channel is determined based on the original task quantity and / or the allocated task quantity corresponding to the transmission channel; the original task quantity is received from the front-end image collector and reflects the task quantity corresponding to the analysis of the image data transmitted through the transmission channel in the first image processing period; and the allocated task quantity is the task quantity allocated to the transmission channel in the last computing round. Therefore, the target task quantity corresponding to the transmission channel in the target computing round is accurately calculated and updated, so as to further guide the calculation strategy in the target computing round. For each transmission channel, the task quantity proportion of the target task quantity relative to the sum of the target task quantities corresponding to each transmission channel is determined, and the target channel is selected from each transmission channel based on the task quantity proportion, so that the target channel with greater data tilt influence is targeted to take the balancing strategy, so as to improve the balancing efficiency. The original task quantity corresponding to the target channel is allocated to all transmission channels and enters the next computing round until the calculation stop condition is met, so that the balanced allocation of the task quantity is realized, thereby solving the situation of data tilt generated in the stage allocated to the reduction node, and only the original task quantity of the same target channel is balanced once, so that when the back-end analyzes the image data of the same transmission channel, only one analysis is needed, thereby avoiding the problem that the original task quantity corresponding to the same target channel is allocated multiple times, and the back-end needs multiple rounds of analysis. In the second image processing period, according to the composition characteristics of the target task quantities corresponding to each transmission channel, the intermediate data processed by the mapping node after the image data transmitted through each transmission channel is allocated to the reduction node, so that the balanced allocation of each computing round is implemented at the reduction node, and load balancing is realized at the reduction node, thereby solving the problem of data tilt.

[0053] Figure 3 A flowchart of an image processing method based on end-cloud cooperation provided for another embodiment of the present application is based on the optimization of the above-mentioned embodiment, and the solutions not described in detail in the present embodiment are described in the above-mentioned embodiment. As shown in Figure 3 The method of the embodiment of the present application specifically includes the following steps:

[0054] S210, in the target computing round, for each transmission channel, the target task quantity corresponding to the transmission channel is determined based on the original task quantity and / or the allocated task quantity corresponding to the transmission channel; the original task quantity is received from the front-end image collector and reflects the task quantity corresponding to the analysis of the image data transmitted through the transmission channel in the first image processing period; and the allocated task quantity is the task quantity allocated to the transmission channel in the last computing round.

[0055] S220, for each transmission channel, determine a task amount proportion of the target task amount relative to a target task amount sum corresponding to each transmission channel, and select a target channel from each transmission channel based on the task amount proportion.

[0056] S230, distribute the original task amount corresponding to the target channel to all transmission channels, and enter the next calculation round until the calculation stop condition is met.

[0057] S240, in the second image processing period, for each transmission channel, count the target channels to which the allocated task amount in the target task amount in the transmission channel belongs.

[0058] For example, in the process of balancing and distributing based on the target task amount corresponding to each transmission channel, for each transmission channel, the target channels to which the allocated task amount in the target task amount in the transmission channel belongs can be counted, that is, it is determined which transmission channels are used as target channels. The target channels to which the allocated task amount belongs can be determined according to the composition characteristics of the target task amount corresponding to the transmission channel. Specifically, if the target task amount corresponding to the transmission channel only includes the allocated task amount and does not include the original task amount, it is determined that the transmission channel is used as the target channel. The original task amount of the transmission channel has been distributed to each transmission channel, and accordingly it can be determined which of the transmission channels are used as target channels.

[0059] S250, average the intermediate data of the image data transmitted by the target channel after being processed by the mapping node and distribute it to each reduction node.

[0060] For example, since the original task amount in the target channel is distributed in the calculation process, accordingly, in the actual application process, the intermediate data of the image data corresponding to the original task amount in the target channel after being processed by the mapping node needs to be distributed to all reduction nodes corresponding to all transmission channels, so as to realize balanced distribution of data and reduce the calculation pressure of the reduction node corresponding to the target channel. In order to achieve balancing as much as possible, in the distribution process, the intermediate data of the image data transmitted by the target channel after being processed by the mapping node can be evenly distributed to each reduction node, so as to avoid the situation that a single reduction node is overloaded in the uneven distribution and it is difficult to effectively realize the effect of balanced processing.

[0061] In the embodiment of the present application, the intermediate data of the image data transmitted by the target channel after being processed by the mapping node is evenly distributed to each reduction node, including: adding a random label to all intermediate data processed by the mapping node after being transmitted by the target channel; and randomly distributing all intermediate data with random labels to each reduction node.

[0062] Exemplarily, in the process of transmitting the image data by the front-end image collector to the mapping node through the transmission channel in the specific implementation process, the image data is normally executed, that is, the intermediate data key-value pair is normally output after the image data is processed by the mapping node, and the key value in the key-value pair is determined according to the identification of the transmission channel corresponding to the image data. In this case, if the conventional scheme is followed, the intermediate data belonging to the same transmission channel will be allocated to the same partition, and the data amount in each partition is skewed. In the embodiment of the present application, before the intermediate data formed by the image data transmitted by the target channel is allocated to the partition corresponding to the reduction node, the random label is attached to the intermediate data, so as to mark that the intermediate data is not directly allocated to the partition corresponding to the target channel, but needs to be randomly allocated to each partition according to the random label, so as to realize the balanced allocation of the intermediate data corresponding to the target channel, that is, the balanced allocation to the reduction node, and the calculation amount of the reduction node corresponding to the target channel is distributed to other reduction nodes, so as to solve the data skew situation.

[0063] In the reduction node, the intermediate data of itself also needs to be corresponding to each transmission channel, and it is clear that the intermediate data is corresponding to which transmission channel of the image data, so the intermediate data needs to be parsed. Since the Key value in the intermediate data is determined according to the identification of the target channel, the identification information of the target channel is still contained in the case of attaching the random label to the intermediate data, so the reduction node can parse the intermediate data with the attached random label to determine the corresponding transmission channel based on this. Specifically, the Key of the intermediate data with the attached random label needs to be calculated by a hash function before being allocated to the reduction node, and the Key of the intermediate data with the attached random label is calculated by the hash function in the reduction node, and the hash value is compared with the received hash value, so as to determine the transmission channel corresponding to the received intermediate data. In the embodiment of the present application, since only the original task amount of the target channel is allocated, the allocated task amount will not be allocated again, that is, only the intermediate data generated by processing the image data transmitted by the target channel is allocated once, and the random label is attached only once, and multiple allocations and multiple random labels are not attached, so as to facilitate the subsequent reduction node to parse and determine the corresponding relationship with the transmission channel, to minimize the parsing complexity, and to solve the data skew situation.

[0064] S260, the intermediate data of the image data transmitted by each transmission channel which is not the target channel and processed by the mapping node is allocated to the reduction node corresponding to the transmission channel.

[0065] Exemplarily, each transmission channel that is not the target channel, that is, all transmission channels except the target channel, is a transmission channel containing the original task amount in terms of the composition of the target task amount. For each transmission channel that is not the target channel, the intermediate data of the image data transmitted through the transmission channel and processed by the mapping node is distributed to the reduction node corresponding to the transmission channel, that is, the intermediate data that does not need to be evenly distributed is distributed according to the normal condition.

[0066] The embodiment of the present application provides a kind of based on end cloud cooperation's image processing method, for each transmission channel, the target channel to which the distribution task amount in the target task amount in the transmission channel belongs is counted;The intermediate data of the image data transmitted by the target channel and processed after mapping node is evenly distributed to each reduction node;The intermediate data of the image data transmitted in each transmission channel that is not the target channel and processed after mapping node is distributed to the reduction node corresponding to the transmission channel.The above scheme can be distributed based on the distribution mode determined in the calculation round in the process actually executed in back end, the intermediate data of the image data transmitted by the target channel and processed in mapping node is evenly distributed to each reduction node, the balanced distribution of data is realized, and since only original task amount is distributed, that is, only one round of distribution is carried out to the intermediate data obtained by processing the image data transmitted by the target channel, secondary distribution is not carried out, the complexity of distribution can be reduced, and the complexity of the reduction node when corresponding intermediate data corresponding to each transmission channel is analyzed.

[0067] Figure 4 A flow chart of a kind of based on end cloud cooperation's image processing method provided by another embodiment of the present application, the above embodiment is optimized based on the above embodiment, and the scheme not described in detail in the embodiment of the present application is described in the above embodiment. As shown in Figure 4 The method of the embodiment of the present application specifically includes the following steps:

[0068] S310, in the target calculation round, for each transmission channel, the target task amount corresponding to the transmission channel is determined based on the original task amount and / or distribution task amount corresponding to the transmission channel;Wherein, the original task amount is received from front-end image collector, and reflects the task amount corresponding to the analysis of the image data transmitted through the transmission channel in the first image processing period;The distribution task amount is the task amount distributed to the transmission channel in the last calculation round.

[0069] S320, for each transmission channel, the task amount proportion of target task amount relative to the sum of target task amount corresponding to each transmission channel is determined.

[0070] For example, for each transmission channel, the target task amount of the transmission channel is determined relative to the task amount proportion of the sum of the task amounts corresponding to each transmission channel. Assuming that the target task amount of channel 1 is 18, the target task amount of channel 2 is 3, and the target task amount of channel 3 is 6, the task amount proportion of channel 1 is 18 / (18+3+6)≈66.7%. The task amount proportion of channel 2 is 3 / (18+3+6)≈11.1%. The task amount proportion of channel 3 is 6 / (18+3+6)≈22.2%.

[0071] In S330, if there is a transmission channel corresponding to a task amount proportion that is the largest and exceeds a preset task amount proportion threshold, the transmission channel is taken as a target channel.

[0072] For example, the task amount proportion reflects the task amount required for analyzing and processing the images transmitted by the transmission channel, that is, reflects the amount of calculation of the reduction node corresponding to the transmission channel for processing and analyzing the image data transmitted by the transmission channel. In order to avoid data skew, the transmission channel corresponding to the task amount proportion that is the largest and exceeds the preset task amount proportion threshold can be taken as the target channel, so as to balance the intermediate data processed by the image data transmitted by the target channel, and realize data balancing in the reduction node.

[0073] In S340, the original task amount corresponding to the target channel is distributed to all transmission channels, and enters the next calculation round.

[0074] In S350, if the task amount proportions corresponding to all transmission channels do not exceed the preset task amount proportion threshold, it is determined that the calculation stop condition is met.

[0075] For example, in each calculation round, after calculating the task amount proportions corresponding to the transmission channels, if the task amount proportions do not exceed the preset task amount proportion threshold, it reflects that the image data transmitted by each transmission channel is balanced, that is, the intermediate data finally distributed to the reduction node is balanced, and the data does not produce skew, so the current calculation round is ended, and the task amount distribution calculation is not continued.

[0076] In a specific implementation, in the first image processing period, since the front-end image collector has not yet been able to count the original task amounts corresponding to each transmission channel, the back-end mapping node and the reduction node directly distribute according to the conventional scheme. After the first image processing period ends, the front-end image collector feeds back the original task amounts corresponding to each transmission channel, for example, the original task amount of channel 1 is 18, the original task amount of channel 2 is 3, and the original task amount of channel 3 is 6.

[0077] In the first computing round, the target task quantity of each transmission channel is determined, the target task quantity of channel 1 is 18, the target task quantity of channel 2 is 3, and the target task quantity of channel 3 is 6. The task quantity proportion of channel 1 is 18 / (18+3+6)≈66.7%. The task quantity proportion of channel 2 is 3 / (18+3+6)≈11.1%. The task quantity proportion of channel 3 is 6 / (18+3+6)≈22.2%. Assuming that the preset task quantity proportion threshold is 40%, the task quantity proportion of channel 1 is the largest and exceeds the preset task quantity proportion threshold, so channel 1 is taken as the target channel, and the original task quantity 18 is evenly distributed to the three transmission channels. After distribution, the target task quantity of channel 1 = the distributed task quantity 18 / 3 = 6. The target task quantity of channel 2 = the original task quantity 3 + the distributed task quantity 18 / 3 = 3+6 = 9. The target task quantity of channel 3 = the original task quantity 6 + the distributed task quantity 18 / 3 = 6+6 = 12.

[0078] In the second computing round, the target task quantity of each transmission channel is determined, the target task quantity of channel 1 is 6, the target task quantity of channel 2 is 9, and the target task quantity of channel 3 is 12. The task quantity proportion of channel 1 is 6 / (6+9+12)≈22.2%. The task quantity proportion of channel 2 is 9 / (6+9+12)≈33.3%. The task quantity proportion of channel 3 is 12 / (6+9+12)≈44.4%. The task quantity proportion of channel 3 exceeds the preset task quantity proportion threshold, so channel 3 is taken as the target channel, and the original task quantity 6 of channel 3 is evenly distributed to the three transmission channels. After distribution, the target task quantity of channel 1 = the distributed task quantity 18 / 3 + the distributed task quantity 6 / 3 = 6+2 = 8. The target task quantity of channel 2 = the original task quantity 3 + the distributed task quantity 18 / 3 + the distributed task quantity 6 / 3 = 3+6+2 = 11. The target task quantity of channel 3 = the distributed task quantity 18 / 3 + the distributed task quantity 6 / 3 = 6+2 = 8.

[0079] In the third computing round, the target task quantity of each transmission channel is determined, the target task quantity of channel 1 is 8, the target task quantity of channel 2 is 11, the target task quantity of channel 3 is 8, and the task quantity proportion of channel 1 is 8 / (8+11+8)≈29.65%. The task quantity proportion of channel 2 is 11 / (8+11+8)≈40.7%. The task quantity proportion of channel 3 is 8 / (8+11+8)≈29.65%. The task quantity proportion of channel 2 exceeds the preset task quantity proportion threshold, channel 2 is taken as the target channel, and the original task quantity 3 of channel 2 is evenly distributed to the three transmission channels. After the distribution is completed, the target task quantity of channel 1 = the distributed task quantity 18 / 3 + the distributed task quantity 6 / 3 + the distributed task quantity 3 / 3 = 6+2+1 = 9. The target task quantity of channel 2 = the distributed task quantity 18 / 3 + the distributed task quantity 6 / 3 + the distributed task quantity 3 / 3 = 6+2+1 = 9. The target task quantity of channel 3 = the distributed task quantity 18 / 3 + the distributed task quantity 6 / 3 + the distributed task quantity 3 / 3 = 6+2+1 = 9.

[0080] In the fourth computing round, the target task quantity of each transmission channel is determined, the target task quantity of channel 1 is 9, the target task quantity of channel 2 is 9, and the target task quantity of channel 3 is 9, the task quantity proportion of channel 1 is 9 / (9+9+9)≈33.3%. The task quantity proportion of channel 2 is 9 / (9+9+9)≈33.3%. The task quantity proportion of channel 3 is 9 / (9+9+9)≈33.3%. The task quantity proportions are all less than the preset task quantity proportion threshold, then the computing round is ended and the distribution is completed.

[0081] In an implementation, after the allocation is completed, the target task amounts corresponding to the respective transmission channels tend to be equalized, and the intermediate data of the image data processed by the corresponding transmission channels is also allocated to the reduction nodes in a balanced manner. In a subsequent image processing cycle, if the difference between the raw task amount of the image collector for the same transmission channel in the image processing cycle and the raw task amount in the previous image processing cycle is less than a preset difference threshold, or for each transmission channel, the ratio of the raw task amount of the transmission channel in the image processing cycle to the sum of the raw task amounts is less than a preset ratio threshold from the ratio of the raw task amount to the sum of the raw task amounts in the previous image processing cycle, the allocation can continue to be performed based on the allocation strategy determined in the second image processing cycle. If the above conditions are not met, the calculation and allocation of each calculation round are triggered. In another implementation, the equalization calculation and allocation can also be performed in each image processing cycle, that is, in the image processing cycle about to be performed, the calculation of each calculation round and the equalization allocation are performed according to the raw task amount of the image data actually transmitted by each transmission channel in the previous image processing cycle, and the intermediate data of the image data of each transmission channel processed by the mapping node is allocated to the reduction node according to the equalization allocation result, so that the equalization allocation is performed according to the actual task amount in each image processing cycle.

[0082] In the embodiments of the present application, the determination process of the preset task amount proportion threshold value includes: taking the ratio of the preset maximum proportion to the number of all transmission channels as a lower threshold value, and taking the ratio of the preset maximum proportion to the number of part of the transmission channels as an upper threshold value; wherein the preset maximum proportion is 100%; the number of the part of the transmission channels is the number of all transmission channels minus one; and the preset task amount proportion threshold value is determined between the lower threshold value and the upper threshold value.

[0083] For example, the ratio of 100% to the total number of transmission channels can be used as a lower threshold value, which ensures that at most one transmission channel has a task quantity ratio exceeding the preset task quantity ratio threshold value in this case, thereby avoiding the situation where the task quantity ratio of a transmission channel exceeds the preset task quantity ratio threshold value even if the allocation is relatively balanced in the calculation round. For example, assuming that the preset task quantity ratio threshold value is set to 30%, the task quantity ratios of two transmission channels are both 35%, the allocation is relatively balanced, and the task quantity ratio of the third transmission channel is 30%, which cannot be further optimized based on the original task quantity allocation. However, the task quantity ratio still exceeds the preset task quantity ratio threshold value. The ratio of 100% to the number of partial transmission channels can be used as an upper threshold value. The number of partial transmission channels is less than the total number of transmission channels. In order to give a more reasonable limit, the number of partial transmission channels can be one less than the total number of transmission channels, thereby avoiding a situation where the preset task quantity ratio threshold value is set too high, resulting in a situation where the task quantity ratio of a transmission channel meets the requirement of the preset task quantity ratio threshold value after allocation, but is still unbalanced. For example, if the preset task quantity threshold value is set to 50% and the task quantity ratio of a transmission channel is 49%, it meets the requirement of being lower than the preset task quantity ratio threshold value. However, the task quantity ratios of the other two transmission channels are 5% and 46% respectively, and there is still a situation of data skew. Between the lower threshold value and the upper threshold value, the preset task quantity ratio threshold value is determined, thereby setting a reasonable preset task quantity ratio threshold value, that is, the balanced allocation that meets the preset task quantity ratio threshold value can be quickly achieved within a reasonable calculation round, and reasonable balancing can be achieved.

[0084] In the embodiments of the present application, after the intermediate data obtained by processing the image data transmitted through each transmission channel by the mapping node is allocated to the reduction node, the method further comprises: determining, for each reduction node, a task skew degree of the reduction node according to the task processing amount of the reduction node and the total sum of the task processing amounts of all reduction nodes; and / or, determining, for each reduction node, a skew degree variance according to the task processing amount of the reduction node and the average value of the task processing amounts of all reduction nodes; and adjusting the preset task quantity ratio threshold value according to the task skew degree and / or the skew degree variance.

[0085] For example, after the intermediate data obtained by processing the image data transmitted through each transmission channel by the mapping node is allocated to the reduction node, the balancing can also be evaluated according to statistical data. Specifically, for each reduction node, a task skew degree of the reduction node is determined according to the task processing amount of the reduction node in the actual task processing process and the task processing amount of all reduction nodes. Let Ti be the task processing amount of the i th Reduce task, T totalThe total amount of task processing of all Reduce tasks. The skewness S of a single Reduce task is i may be expressed as: S i = T i / T total . And / or, for each reduction node, the skewness variance is determined according to the task processing amount of the reduction node and the average of the task processing amounts of all reduction nodes. Let T avg be the average of the task processing amounts of all Reduce tasks, and n be the total number of Reduce tasks. That is:

[0086] T avg= T total / n;

[0087] The overall skewness may be expressed by variance, that is, the sum of squares of the difference between the task processing amount of each Reduce task and the average of the task processing amounts divided by the total number n of Reduce tasks:

[0088] ;

[0089] Both the task skewness and the skewness variance can reflect the balance of the task processing amount in the actual task processing process of the reduction node. Specifically, if the difference between the task skewness of each reduction node exceeds the preset skewness difference value, it reflects that the task processing amount between the reduction nodes is still unbalanced. Similarly, if the skewness variance exceeds the preset skewness variance threshold, it reflects that the skewness variance is large, that is, the fluctuation of the task processing amount in each reduction node relative to the average is large, reflecting that the task processing amount in each reduction node is large, and the reduction node still exists unbalanced. If it is determined that there is still an unbalanced situation of the task processing amount in the reduction node according to the task skewness and / or the skewness variance, the preset task amount proportion threshold can be adjusted to more strictly perform the subsequent data balancing strategy.

[0090] In the embodiments of the present application, adjusting the preset task amount proportion threshold according to the task skewness and / or the skewness variance includes: if the statistical data value of the task skewness exceeds the preset task amount proportion threshold, and / or the skewness variance exceeds the preset variance threshold, the preset task amount proportion threshold is reduced; the method further includes: if the calculation round exceeds the preset round, the preset task amount proportion threshold is increased.

[0091] For example, if the statistical value of the task skew exceeds the preset task proportion threshold, and / or the skew variance exceeds the preset variance threshold, it reflects that the task processing amount in the reduction node is still unbalanced. In order to achieve more balanced, the preset task proportion threshold can be adaptively reduced, so as to improve the condition of meeting data balance, and realize a more strict balanced allocation strategy.

[0092] In the embodiment of the application, if the calculation round exceeds the preset round, it reflects that the calculation consumption time is relatively long. In order to improve efficiency and real-time performance, the preset task proportion threshold can be adaptively increased, so as to reduce the calculation time consumption and improve the real-time performance of the backend processing under the condition of meeting the basic balance. The preset round can be determined according to the actual situation, and specifically can be positively correlated with the number of transmission channels, that is, the more the number of transmission channels, the more the allowed calculation rounds of iteration.

[0093] In the above process of adjusting the preset task proportion threshold, the preset task proportion threshold needs to be adjusted between the lower threshold and the upper threshold.

[0094] S360, in the second image processing period, according to the composition characteristics of the target task amount corresponding to each transmission channel, the intermediate data processed by the mapping node after the image data transmitted through each transmission channel is distributed to the reduction node; wherein the first image processing period is located before the second image processing period.

[0095] The embodiment of the application provides an image processing method based on end-cloud cooperation. For each transmission channel, the task proportion of the target task amount relative to the sum of the target task amounts corresponding to each transmission channel is determined. If there is a transmission channel corresponding to the maximum task proportion and exceeding the preset task proportion threshold, the transmission channel is taken as a target channel, and the original task amount corresponding to the target channel is distributed to all transmission channels, so as to realize the balanced allocation of data corresponding to the target channel with more task proportion, solve the data tilt situation, and only divide the original task amount, reduce the complexity of the backend analysis. If the task proportions corresponding to all transmission channels do not exceed the preset task proportion threshold, it is determined that the calculation stop condition is met, so as to realize the quantitative detection of whether the data is balanced, and stop the calculation in time under the condition of balanced data distribution, so as to facilitate the real-time business processing of the backend.

[0096] Figure 5 A flowchart of an image processing method based on end-cloud cooperation is provided for another embodiment of the application. The embodiment of the application is optimized based on the above-mentioned embodiment, and the schemes not described in detail in the embodiment of the application are described in the above-mentioned embodiment. As shown in Figure 5 The method of the embodiment of the application specifically includes the following steps:

[0097] S410, in the target computing round, for each transmission channel, determining a target task quantity corresponding to the transmission channel based on a raw task quantity and / or an allocated task quantity corresponding to the transmission channel; wherein the raw task quantity is received from a front-end image collector and reflects a task quantity corresponding to analyzing image data transmitted through the transmission channel in a first image processing period; and the allocated task quantity is a task quantity allocated to the transmission channel in a previous computing round.

[0098] S420, for each transmission channel, determining a task quantity proportion of the target task quantity relative to a sum of target task quantities corresponding to each transmission channel, and selecting a target channel from each transmission channel based on the task quantity proportion.

[0099] S430, distributing the raw task quantity corresponding to the target channel to all transmission channels, and entering a next computing round until a computing stop condition is met.

[0100] S440, in a second image processing period, according to the composition characteristics of the target task quantity corresponding to each transmission channel, distributing intermediate data processed by a mapping node to a reduction node, wherein the first image processing period is before the second image processing period.

[0101] S450, if it is detected that at least two transmission channels correspond to the same front-end image collector, sending task quantity self-balancing signaling to the front-end image collector, and feeding back an average task processing quantity of the reduction node, so as to trigger the front-end image collector to calculate a required number of transmission channels based on a sum of raw task quantities corresponding to each transmission channel and the average task processing quantity through the task quantity self-balancing signaling, and to distribute image data containing the average task processing quantity to each transmission channel.

[0102] In the embodiments of the present application, if the backend server detects that at least two transmission channels correspond to the same front-end image collector, the front-end image collector can be combined to cooperatively perform the equalization distribution of image data. In this case, the front-end image collector is a multi-channel image collector, such as a current fisheye camera, a hawk-eye camera, a binocular image collector, a multi-lens panoramic image collector, etc. Specifically, task quantity self-balancing signaling can be sent to the front-end image collector, and the average task processing amount of the reduction node is fed back, and the front-end image collector triggers the execution of the task quantity self-balancing task after receiving the task quantity self-balancing signaling. Specifically, based on the sum of the original task amounts corresponding to each transmission channel and the average task processing amount, the number of transmission nodes corresponding to the backend reduction nodes, that is, the number of required transmission channels, is calculated, that is, when the image data transmitted through the transmission channel is distributed to the corresponding reduction node to make the task processing amount of the reduction node meet the average task processing amount, the image data of the front-end image collector needs to be transmitted through several transmission channels respectively. After calculating the required number of transmission channels, the image data containing the average task processing amount is distributed to each transmission channel to transmit the image data to different reduction nodes through different transmission channels, so that different reduction nodes realize the processing of the average task processing amount of the image data, and realize the data balance of the backend with the assistance of the front-end image collector.

[0103] In the embodiments of the present application, the task quantity self-balancing signaling triggers the front-end image collector to calculate the required channel number based on the sum of the original task amounts corresponding to each transmission channel and the average task processing amount, and distribute the image data containing the average task processing amount to each transmission channel, including: taking the ratio of the sum of the original task amounts to the average task processing amount as the required channel number; for the required channel number of transmission channels, the task quantity self-balancing signaling triggers the front-end image collector to scan in the collected picture, and when the required analysis task amount of the target appearing in the continuous picture area reaches the average task processing amount, the image data in the picture area is distributed to a transmission channel for transmission until all transmission channels are distributed.

[0104] Exemplarily, a ratio of a total of original task amounts of all transmission channels corresponding to the same front-end image collector to an average task processing amount can be taken as the number of demand channels. For the number of demand channels of transmission channels, through task amount self-balancing signaling, the front-end image collector is triggered to scan in the entire collection picture, and when a required task amount of a target appearing in a continuous picture region reaches the average task processing amount, image data in the picture region is allocated to a transmission channel for transmission until all transmission channels are allocated. That is, each picture region in the collection picture is partitioned according to a required analysis task amount of the target appearing, and continuous picture regions in which the required analysis task amount of the target appearing reaches the average task processing amount are uniformly allocated to the same transmission channel for transmission.

[0105] In a specific implementation manner, a video stream of the fisheye camera has multiple settable PTZ transmission channels, such as Figure 6 The fisheye camera is shown in the whole collection picture, such as Figure 7 The picture corresponding to the four PTZ transmission channels. For example, the fisheye camera is installed in each large supermarket, and when a discount promotion activity is carried out in the supermarket, the number of people in the promotion area can be much larger than that in other sales areas (the first quadrant in the collection picture contains 27 task amounts, and the other three quadrants each contain 3 task amounts). After receiving the task amount self-balancing signaling from the cloud server, the fisheye camera analyzes and obtains the average task processing amount 9, and then calculates (27+3+3+3) / 9=4, that is, four PTZ transmission channels are also needed, and the optimal solution of each transmission channel is to contain 9 targets with the average task processing amount. As shown in Figure 8 Specifically, the fisheye camera scans counterclockwise from the X axis, and after the task amount in the scanning region S1 is 9, the first PTZ channel is divided, and then the scanning counterclockwise is continued to obtain the task amount 9 in the scanning region S2, and the second PTZ channel is divided, and then the scanning counterclockwise is continued to obtain the task amount 9 in the scanning region S3, and the third PTZ channel is divided, and finally the regions S4 of the three quadrants are merged into the fourth PTZ channel because the task amount is small. In this way, after the next image processing period T+1 arrives, the video stream of each channel of the fisheye camera in the above PTZ electronic channel is transmitted. Under the premise that the business amount in the front and rear periods is relatively stable, the analysis task amount in the channel ID of the front-end camera is self-balanced and optimized, the front-end shares the load of multiple rounds of balance calculation in the Map partitioning stage of the cloud server, and the calculation time is shortened.

[0106] Suppose that the promotion in the supermarket ends, and the number of people in each quadrant of the fisheye camera is small, such as 8 task amounts in total, which is close to the average task amount 9, and the demand channel storage is 8 / 9≈1. At this time, the fisheye camera does not need to maintain four PTZ transmission channels to access the cloud, and can be merged into one PTZ transmission channel to send a video stream for analysis task of the cloud server.

[0107] In the embodiment of the present application, the task amount self-balancing signaling triggers the front-end image collector to calculate the required channel number based on the sum of the original task amount corresponding to each transmission channel and the average task processing amount, and to allocate image data containing the average task processing amount to each transmission channel. In addition, image data of a partial region of interest of a user can be allocated to different transmission channels for transmission, instead of scanning and transmitting all captured images, which not only meets the user's demand, but also reduces transmission pressure and unnecessary bandwidth waste. For example, in the overall captured image of a camera, a user only pays attention to the first quadrant and the third quadrant of the image region. Therefore, the first quadrant of the image region can be scanned from one end point, and the task amount of target analysis is accumulated whenever the target is scanned. When the sum of the task amount of target analysis scanned reaches the average task amount, the image data corresponding to the scanned image region is allocated to a transmission channel for transmission. The scanning continues from zero task amount, and the first quadrant and the third quadrant of the image region are completely scanned and allocated to each transmission channel for image data transmission, thereby completing the front-end image collector pre-balancing allocation process.

[0108] The embodiment of the present application provides an image processing method based on end-cloud cooperation. If it is detected that at least two transmission channels correspond to the same front-end image collector, task amount self-balancing signaling is sent to the front-end image collector, and the average task processing amount of the reduction node is fed back, so as to trigger the front-end image collector to calculate the required channel number based on the sum of the original task amount corresponding to each transmission channel and the average task processing amount, and to allocate image data containing the average task processing amount to each transmission channel. Through the cloud cooperation of the front-end image collector and the cloud server, balanced processing of data is realized. The front-end image collector performs pre-data balancing allocation, so that the image data transmitted to the back end tends to be balanced. Therefore, in the balancing process in the back end, the calculation round is reduced, the efficiency and real-time performance of the back-end balancing are improved, and more efficient and effective balancing is realized.

[0109] Figure 9 A structure diagram of an image processing device based on end-cloud cooperation is provided in the embodiment of the present application. The device can execute the image processing method based on end-cloud cooperation provided in any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method. Figure 9 As shown in the figure, the device comprises:

[0110] The target task amount determination module 510 is configured to determine, for each transmission channel, a target task amount corresponding to the transmission channel based on an original task amount and / or an allocated task amount corresponding to the transmission channel in a target calculation round, wherein the original task amount is received from a front-end image collector and reflects a task amount corresponding to analysis of image data transmitted through the transmission channel in a first image processing period; and the allocated task amount is a task amount allocated to the transmission channel in a previous calculation round.

[0111] The target channel selection module 520 is configured to determine, for each transmission channel, a task amount proportion of the target task amount relative to a sum of target task amounts of all transmission channels, and select a target channel from the transmission channels based on the task amount proportion.

[0112] The calculation module 530 is configured to allocate the original task amount corresponding to the target channel to all transmission channels and enter a next calculation round until a calculation stop condition is met.

[0113] The allocation module 540 is configured to allocate, in a second image processing period, intermediate data obtained by processing image data transmitted through each transmission channel by a mapping node to a reduction node according to a composition feature of the target task amount corresponding to each transmission channel, wherein the first image processing period is before the second image processing period.

[0114] In the embodiment of the present application, the target channel selection module 520 allocates, according to a composition feature of the target task amount corresponding to each transmission channel, intermediate data obtained by processing image data transmitted through each transmission channel by a mapping node to a reduction node, including: for each transmission channel, counting target channels to which the allocated task amount in the target task amount of the transmission channel belongs; evenly allocating intermediate data obtained by processing image data transmitted through the target channel by the mapping node to each reduction node; and allocating intermediate data obtained by processing image data transmitted through each transmission channel that is not a target channel by the mapping node to a reduction node corresponding to the transmission channel.

[0115] In the embodiment of the present application, the target channel selection module 520 evenly allocates intermediate data obtained by processing image data transmitted through the target channel by the mapping node to each reduction node, including: adding random labels to all intermediate data obtained by processing image data transmitted through the target channel by the mapping node; and randomly allocating all intermediate data with random labels to each reduction node.

[0116] In the embodiment of the present application, the target channel selection module 520 selects a target channel from each transmission channel based on the task amount proportion, including: if there is a transmission channel corresponding to the task amount proportion that is the largest and exceeds the preset task amount proportion threshold, the transmission channel is taken as the target channel; correspondingly, the calculation stop condition includes: if the task amount proportions corresponding to all transmission channels do not exceed the preset task amount proportion threshold, it is determined that the calculation stop condition is met.

[0117] In the embodiment of the present application, the device includes: a threshold determination module, configured to take the ratio of the preset maximum proportion to the number of all transmission channels as a lower threshold, and take the ratio of the preset maximum proportion to the number of part of the transmission channels as an upper threshold; wherein the preset maximum proportion is 100%; the number of the part of the transmission channels is the number of all transmission channels minus one; the preset task amount proportion threshold is determined between the lower threshold and the upper threshold.

[0118] In the embodiment of the present application, after the intermediate data processed by the mapping node after the image data transmitted through each transmission channel is distributed to the reduction node, the device further includes an adjustment module, configured to: for each reduction node, determine the task inclination of the reduction node according to the task processing amount of the reduction node and the total task processing amount of all reduction nodes; and / or, for each reduction node, determine the inclination variance according to the task processing amount of the reduction node and the average task processing amount of all reduction nodes; and adjust the preset task amount proportion threshold according to the task inclination and / or inclination variance.

[0119] In the embodiment of the present application, the adjustment module adjusts the preset task amount proportion threshold according to the task inclination and / or inclination variance, including: if the statistical data value of the task inclination exceeds the preset task amount proportion threshold, and / or the inclination variance exceeds the preset variance threshold, the preset task amount proportion threshold is reduced; the method further includes: if the calculation round exceeds the preset round, the preset task amount proportion threshold is increased.

[0120] In the embodiment of the present application, the device further includes a triggering module, configured to: if it is detected that at least two transmission channels correspond to the same front-end image collector, send task amount self-balancing signaling to the front-end image collector, and feed back the average task processing amount of the reduction node, so as to trigger the front-end image collector to calculate the number of required channels based on the total original task amount corresponding to each transmission channel and the average task processing amount through the task amount self-balancing signaling, and distribute image data containing the average task processing amount to each transmission channel.

[0121] In the embodiment of the present application, the triggering module triggers the front-end image collector to calculate the required channel number based on the original task amount sum of each transmission channel and the average task processing amount through the task amount self-balancing signaling, and allocates image data containing the average task processing amount to each transmission channel, including: taking the ratio of the original task amount sum and the average task processing amount as the required channel number; for the required channel number of transmission channels, triggering the front-end image collector to scan in the collected picture through the task amount self-balancing signaling, and when the required analysis task amount of the target appearing in the continuous picture area reaches the average task processing amount, the image data in the picture area is allocated to a transmission channel for transmission until all transmission channels are allocated.

[0122] The image processing device based on end-cloud cooperation provided in the embodiment of the present application can execute the image processing method based on end-cloud cooperation provided in any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0123] Figure 10 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0124] As shown in Figure 10 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0125] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless cloud-based end collaboration image processing transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0126] The processor 11 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the cloud-based end collaboration image processing method.

[0127] In some embodiments, the cloud-based end collaboration image processing method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the cloud-based end collaboration image processing method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the cloud-based end collaboration image processing method by any other appropriate means, such as by means of firmware.

[0128] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0129] A computer program for implementing the methods of the present application can be written in any combination of one or more programming languages. The computer program can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0130] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store the computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0131] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0132] The systems and techniques described here can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here, or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0133] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0134] The embodiment of the present application further provides a computer program product, comprising a computer program which, when executed by a processor, implements the image processing method based on end-cloud cooperation as provided in any embodiment of the present application.

[0135] The computer program code implementing the application can be written in one or more programming languages or combinations of languages including object oriented languages such as Java, Smalltalk, C++ or conventional procedural programming languages such as "C" or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0136] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in this application can be performed in parallel, in series, or in a different order, as long as the desired information is achieved by the technical solution of the present application, which is not limited herein.

[0137] The foregoing detailed description has not been a limitation on the scope of the application. It will be apparent to those skilled in the art that modifications, combinations, sub-combinations and alternatives can be made to the specific embodiments disclosed herein without departing from the spirit and principles of the application. Any further modifications, changes or improvements made to the specific embodiments described herein are to be regarded as being within the scope of the application.

Claims

1. An image processing method based on end-cloud cooperation, characterized in that, The method comprises: in a target computing round, for each transmission channel, determining a target task quantity corresponding to the transmission channel based on an original task quantity and / or an allocated task quantity corresponding to the transmission channel; wherein the original task quantity is received from a front-end image collector and reflects a task quantity corresponding to analysis of image data transmitted through the transmission channel in a first image processing period; the allocated task quantity is a task quantity allocated to the transmission channel in a previous computing round; for each transmission channel, determining a task quantity proportion of the target task quantity relative to a sum of target task quantities corresponding to each transmission channel, and selecting a target channel from each transmission channel based on the task quantity proportion; allocating the original task quantity corresponding to the target channel to all transmission channels, and entering a next computing round until a computing stop condition is met; wherein the computing stop condition comprises that the task quantity proportion of each transmission channel is less than a threshold value, or the difference between the task quantity proportions of each transmission channel is less than another threshold value; in a second image processing period, according to the composition characteristics of the target task quantity corresponding to each transmission channel, distributing intermediate data processed by a mapping node to a reduction node, wherein the first image processing period is before the second image processing period; distributing intermediate data processed by a mapping node to a reduction node according to the composition characteristics of the target task quantity corresponding to each transmission channel, comprises: for each transmission channel, counting target channels to which the allocated task quantity in the target task quantity belongs; distributing intermediate data processed by a mapping node to each reduction node; distributing intermediate data processed by a mapping node to the reduction node corresponding to each transmission channel.

2. The method of claim 1, wherein, distributing intermediate data processed by a mapping node to each reduction node, comprises: appending random labels to all intermediate data processed by the mapping node and transmitted through the target channel; randomly distributing all intermediate data with random labels to each reduction node.

3. The method of claim 1, wherein, selecting a target channel from each transmission channel based on the task quantity proportion, comprises: if there is a transmission channel corresponding to a maximum task quantity proportion that exceeds a preset task quantity proportion threshold value, the transmission channel is selected as the target channel; correspondingly, the computing stop condition comprises: if the task quantity proportions corresponding to all transmission channels do not exceed the preset task quantity proportion threshold value, it is determined that the computing stop condition is met.

4. The method of claim 3, wherein, the determination process of the preset task quantity proportion threshold value comprises: taking the ratio of a preset maximum proportion to the number of all transmission channels as a lower threshold value, and taking the ratio of the preset maximum proportion to the number of part of the transmission channels as an upper threshold value; wherein the preset maximum proportion is one hundred percent; the number of the part of the transmission channels is the number of all transmission channels minus one; determining the preset task quantity proportion threshold value between the lower threshold value and the upper threshold value.

5. The method of claim 3, wherein, After the intermediate data of the image data transmitted through each transmission channel and processed by the mapping node is distributed to the reduction node, the method further comprises: For each reduction node, determining a task skew of the reduction node according to a task processing amount of the reduction node and a sum of task processing amounts of all reduction nodes; and / or, For each reduction node, determining a skew variance according to a task processing amount of the reduction node and a mean of task processing amounts of all reduction nodes; Adjusting the preset task amount proportion threshold according to the task skew and / or the skew variance.

6. The method of claim 5, wherein, Adjusting the preset task amount proportion threshold according to the task skew and / or the skew variance comprises: If the statistical data value of the task skew exceeds the preset task amount proportion threshold, and / or the skew variance exceeds a preset variance threshold, then the preset task amount proportion threshold is reduced; The method further comprises: If the calculation round exceeds a preset round, then the preset task amount proportion threshold is increased.

7. The method of claim 1, wherein, The method further comprises: If at least two transmission channels correspond to the same front-end image collector is detected, then task amount self-balancing signaling is sent to the front-end image collector, and the average task processing amount of the reduction node is fed back, so that the front-end image collector calculates the required channel number based on the sum of original task amounts corresponding to each transmission channel and the average task processing amount through the task amount self-balancing signaling, and allocates image data containing the average task processing amount to each transmission channel.

8. The method of claim 7, wherein, Triggering the front-end image collector to calculate the required channel number based on the sum of original task amounts corresponding to each transmission channel and the average task processing amount through the task amount self-balancing signaling, and allocating image data containing the average task processing amount to each transmission channel, comprises: Taking the ratio of the sum of original task amounts to the average task processing amount as the required channel number; For the required channel number of transmission channels, triggering the front-end image collector to scan in the collected picture through the task amount self-balancing signaling, and when the required analysis task amount of the target appearing in the continuous picture area reaches the average task processing amount, allocating the image data in the picture area to one transmission channel for transmission until all transmission channels are allocated.

9. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the image processing method based on end-cloud cooperation according to any one of claims 1-8.

Images