Cloud-edge collaborative video processing method and device, medium and equipment

Abstract

The application discloses a cloud-edge cooperative video processing method and device, a medium and equipment, and relates to the field of Internet of Things. The method comprises the following steps: a cloud center node acquires a video processing task, and determines a task message to be delivered according to the video processing task; the cloud center node delivers the task message to a video processing server of an edge node through a message access gateway; the video processing server determines corresponding task data according to the task message, and reports the task data to the cloud center node; and the cloud center node processes the video processing task according to the task data. The scheme provided in the application executes the video processing task through cloud-edge cooperation, improves the real-time performance and processing efficiency of video processing, reduces the uplink bandwidth cost, and reduces the load of the processing cluster and storage cluster of the cloud center node.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of Internet of Things, and in particular to a cloud-edge collaborative video processing method and device, medium and equipment. BACKGROUND

[0002] Artificial intelligence (AI) is a comprehensive technology of computer science, which makes machines have the functions of perception, reasoning and decision-making by studying the design principles and implementation methods of various intelligent machines. Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, such as natural language processing, machine learning, deep learning, etc. Rapid development of artificial intelligence is the key to unlocking the great potential of the Internet of Things. Combined with artificial intelligence, the Internet of Things can be more intelligent and more efficient, bringing unlimited development to the Internet of Things.

[0003] In the field of Internet of Things, video is a kind of data that needs to be collected, transmitted and analyzed in large quantities. With the development of technology, the pixel of camera is getting higher and higher, and the data generated by a single camera will be larger and larger. At the same time, the gradual landing of smart city and smart building will also lead to a larger and larger density of cameras. Both of these will lead to an exponential increase in the amount of video data, testing the data processing capacity and network transmission capacity. SUMMARY

[0004] In order to improve the processing efficiency of video data, the present application provides a cloud-edge collaborative video processing method, device, medium and equipment. The technical solution is as follows:

[0005] In a first aspect, the present application provides a cloud-edge collaborative video processing method, which comprises:

[0006] The cloud center node acquires a video processing task, and determines a task message to be issued according to the video processing task;

[0007] The cloud center node issues the task message to the video processing server of the edge node through the message access gateway;

[0008] The video processing server determines corresponding task data according to the task message, and reports the task data to the cloud center node;

[0009] The cloud center node processes the video processing task according to the task data.

[0010] In a second aspect, the present application provides a cloud-edge collaborative video processing device, which comprises:

[0011] A first task processing module is configured to acquire a video processing task by a cloud center node, and determine a task message to be issued according to the video processing task;

[0012] a message issuing module, configured to issue, by the cloud center node through a message access gateway, the task message to a video processing server of the edge node;

[0013] a second task processing module, configured to determine, by the video processing server, corresponding task data according to the task message, and report the task data to the cloud center node;

[0014] a third task processing module, configured to process, by the cloud center node, the video processing task according to the task data.

[0015] In a third aspect, the present application provides a computer readable storage medium, the computer readable storage medium stores at least one instruction or at least one program, the at least one instruction or at least one program is loaded and executed by a processor to implement the cloud-edge collaborative video processing method according to the first aspect.

[0016] In a fourth aspect, the present application provides a computer device, the computer device includes a processor and a memory, the memory stores at least one instruction or at least one program, the at least one instruction or at least one program is loaded and executed by the processor to implement the cloud-edge collaborative video processing method according to the first aspect.

[0017] In a fifth aspect, the present application provides a computer program product or a computer program, the computer program product or the computer program includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the cloud-edge collaborative video processing method according to the first aspect.

[0018] The cloud-edge collaborative video processing method, device, equipment and storage medium provided by the present application have the following technical effects:

[0019] The scheme provided by the present application replaces the edge video gateway with a video processing server at the edge end, enhances the data processing capability of the edge end, adds a message access gateway at the cloud end, issues a video processing task, and further processes the processing result of the edge end at the cloud end, which can improve the efficiency of video data processing. The cloud-edge collaboration processes the video task, reduces the load of the uplink bandwidth, the center side video processing cluster and the center side historical video storage cluster, and reduces the bandwidth cost, the storage cost and the cluster operation cost.

[0020] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, and the advantages thereof, simple introductions to the drawings needed to be used in the embodiments or prior art descriptions will be given below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those of ordinary skill in the art without any creative effort based on these drawings.

[0022] Figure 1 is an implementation environment schematic diagram of a cloud-edge collaborative video processing method provided by an embodiment of the present application;

[0023] Figure 2 is a flow schematic diagram of a cloud-edge collaborative video processing method provided by an embodiment of the present application;

[0024] Figure 3 is a flow schematic diagram of generating a task message provided by an embodiment of the present application;

[0025] Figure 4 is another flow schematic diagram of generating a task message provided by an embodiment of the present application;

[0026] Figure 5 is a flow schematic diagram of an edge node operating according to a task message provided by an embodiment of the present application;

[0027] Figure 6 is a flow schematic diagram of a processing process of an edge node in a specific scenario provided by an embodiment of the present application;

[0028] Figure 7 is a flow schematic diagram of a processing process of a cloud center node in a specific scenario provided by an embodiment of the present application;

[0029] FIG. 8(1) is a schematic diagram of a traditional architecture based on a client / server in a video monitoring application provided by an embodiment of the present application;

[0030] FIG. 8(2) is a schematic diagram of an architecture based on a message access gateway and an edge node video processing server in a video monitoring application provided by an embodiment of the present application;

[0031] Figure 9 is a processing timing diagram of a video data application task provided by an embodiment of the present application;

[0032] Figure 10 is a processing timing diagram of a video viewing task provided by an embodiment of the present application;

[0033] Figure 11 is a processing timing diagram of a video backup task provided by an embodiment of the present application;

[0034] Figure 12 FIG. 1 is a structural schematic diagram of a cloud-edge collaborative video processing apparatus provided by an embodiment of the present application.

[0035] Figure 13 FIG. 2 is a hardware structural schematic diagram of an apparatus for implementing a method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0036] Artificial intelligence (AI) is the theory, method, technology and application system for using a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain optimal results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making.

[0037] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc.

[0038] The scheme provided by the embodiments of the present application relates to computer vision technology of artificial intelligence, etc.

[0039] Computer vision technology (CV): Computer vision is a science that studies how to make machines "see". Further, it refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further perform image processing to make computer processing more suitable for human eye observation or image transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, etc. It also includes common face recognition, fingerprint recognition and other biometric identification technologies. In the field of Internet of Things, for specific needs, video data may need to be processed for image recognition and portrait search.

[0040] The scheme provided by the embodiments of the present application relates to the field of Internet of Things, and the Internet of Things (IOT) refers to real-time collection of any object or process that needs to be monitored, connected and interacted through various devices and technologies such as information sensors, radio frequency identification technology, global positioning system, infrared sensors, laser scanners and the like, collecting various needed information such as sound, light, heat, electricity, mechanics, chemistry, biology, position and the like, and realizing ubiquitous connection of things and people through various possible network access, and realizing intelligent perception, identification and management of objects and processes. The Internet of Things is an information carrier based on the Internet, a traditional telecommunications network and the like, which enables all ordinary physical objects that can be independently addressed to form an interconnected network.

[0041] The scheme provided by the embodiments of the present application is deployed in the cloud and the edge, and further relates to cloud technology, cloud computing and edge computing and the like.

[0042] Cloud technology: refers to a kind of hosting technology that unifies a series of resources such as hardware, software and network in a wide area network or a local area network to realize data calculation, storage, processing and sharing, and can also be understood as a general term of network technology, information technology, integration technology, management platform technology and application technology based on cloud computing business model application, which can form a resource pool, and be used on demand, flexibly and conveniently. The background service of a technical network system needs a large amount of computing and storage resources, such as video websites, picture websites and more portals, with the high development and application of the Internet industry, every object may have its own identification mark in the future, and needs to be transmitted to the background system for logical processing, and different levels of data will be processed separately, and various industry data need strong system backup, therefore, cloud technology needs to be supported by cloud computing.

[0043] Cloud computing is a computing mode, which distributes computing tasks on a resource pool composed of a large number of computing machines, so that various application systems can obtain computing power, storage space and information services according to needs. The network providing resources is called "cloud". The resources in the "cloud" can be infinitely expanded in the eyes of the user, and can be obtained at any time, used on demand, expanded at any time, and paid according to use. As a basic ability provider of cloud computing, a cloud computing resource pool platform, referred to as a cloud platform, is generally called Infrastructure as a Service (IaaS), and a plurality of types of virtual resources are deployed in the resource pool for external customers to select and use. The cloud computing resource pool mainly includes: computing devices (which can be virtualized machines, including operating systems), storage devices and network devices.

[0044] Edge computing refers to an open platform integrating network, computing, storage and application core capabilities on the side close to the object or data source, to provide near-end services. The application program is initiated on the edge side to produce faster network service response, and meet the basic needs of industry in real-time business, application intelligence, security and privacy protection. Edge computing is between physical entity and industrial connection, or at the top of physical entity. While cloud computing can still access historical data of edge computing.

[0045] To improve the processing efficiency of video data, the embodiments of the present application provide a cloud-edge collaborative video processing method, device, medium and equipment. The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application. The examples of the embodiments are shown in the drawings, wherein the same or similar reference signs represent the same or similar elements or elements with the same or similar functions throughout.

[0046] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0047] Please refer to Figure 1 , which is an implementation environment diagram of a cloud-edge collaborative video processing method provided by the embodiments of the present application, as shown in Figure 1 , the implementation environment can at least include a client 01 and a server 02.

[0048] Specifically, the client 01 can include a smart phone, a desktop computer, a tablet computer, a notebook computer, a digital assistant, a smart wearable device, a monitoring device, a voice interaction device, and the like, and can also include a software running in the device, such as a web page provided by a service provider to a user, and an application provided by the service provider to the user. Specifically, the client 01 is a front-end device of a cloud center node, and can be used to receive an operation of a user and obtain a video processing task issued by the user, such as a triggered video data processing application task, a push stream instruction for video viewing, or a backup rule for video backup. The client 01 can also be used to display a final processing result of the video processing task, such as a current real-time monitoring video or a video recognition result.

[0049] Specifically, the server 02 can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms. The server 02 can include a network communication unit, a processor, a memory, and the like. The terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application. Specifically, the server 02 constitutes an architecture for processing a video task in cooperation with the edge, including a video processing server, a gateway server, and a storage server, and can provide services such as application processing of video data, real-time viewing of video, and backup storage of video.

[0050] The embodiments of the present application can also be implemented in combination with cloud technology. Cloud technology refers to a kind of hosting technology that unifies hardware, software, and network resources in a wide area network or local area network to realize data calculation, storage, processing, and sharing. It can also be understood as a general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. Cloud technology needs to be supported by cloud computing. Cloud computing is a computing mode that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space, and information services as needed. The network that provides resources is called "cloud". Specifically, the server 02 and the database are located in the cloud, and the server 02 can be a physical machine or a virtual machine.

[0051] A cloud-edge collaborative video processing method provided by the present application is introduced below. Figure 2is a flowchart of a cloud-edge collaborative video processing method provided by an embodiment of the present application. The present application provides method operation steps as described in the embodiments or flowcharts, but more or fewer operation steps can be included based on conventional or non-inventive labor. The order of steps listed in the embodiments is only one of the many step execution orders, and does not represent the only execution order. In actual system or server product execution, the method order shown in the embodiments or the drawings can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment). Please refer to Figure 2 The cloud-edge collaborative video processing method provided by an embodiment of the present application can include the following steps:

[0052] S210: The cloud center node obtains a video processing task and determines a task message to be issued according to the video processing task.

[0053] In the embodiments of the present application, the cloud center node can include a message access gateway, a video access gateway, a video processing cluster, a video storage cluster, and a display and control module, wherein the message access gateway and the video access gateway are connected to the cloud center node and the edge node, and the message access gateway only transmits message data between the two, while the video stream data is transmitted through the video access gateway; wherein the display and control module acquires tasks, and the user's client can access the display and control module through a network application or a local application to obtain a video processing task. In the embodiments of the present application, compared with real-time uploading of video data to the cloud and real-time processing, the video is processed in a task triggering mode, which greatly reduces the transmission pressure of the uplink bandwidth and reduces the processing cluster load of the cloud center node and the cost of the storage cluster. In the embodiments of the present application, the video processing task can include a video data application task, a video viewing task, and a video backup task, wherein the video data application task belongs to a task in which the video is used as a data source and the processing logic is not fixed, such as a portrait search task, a license plate search task, a people flow statistics task, and a vehicle flow statistics task, and the video viewing task and the video backup task are regular operations on the video in the Internet of Things.

[0054] In the method provided in the embodiments of the present application, a part of the video processing logic is sunk to the edge node, and the localization of the video processing is realized by fusing the video system of the edge node, thereby improving the real-time performance of the video processing. Therefore, for the video processing task obtained by the cloud center node, the part of the processing task undertaken by each edge node needs to be determined, and the task message is notified to each edge node.

[0055] In the embodiments of the present application, as shown in Figure 3 The method of determining the task message to be issued according to the video processing task can include the following steps:

[0056] S310: determining one or more edge nodes according to the video processing task.

[0057] In the embodiments of the present application, for a video processing task, it is necessary to first determine the edge node that executes the processing task. The edge nodes are usually grouped by labels when deployed, and the labels can represent regional information, data types that can be processed, etc. of the edge nodes. The label of the edge node can be determined according to the video processing task information. For example, according to the region range specified in the video processing task, the edge nodes in the region range are determined.

[0058] S330: determining a task message corresponding to each of the one or more edge nodes according to the video processing task and the one or more edge nodes.

[0059] In the embodiments of the present application, the edge nodes can be all nodes or part of the nodes, and the specific nodes can be determined according to the video processing task. The task content required to be undertaken by different edge nodes can be the same or different, and therefore the task messages can also be different. In addition, in the present application, the task message refers to a transmission signal containing task information, and the network address of the edge node and other information are also carried in the task message.

[0060] Specifically, as shown in Figure 4 When the video processing task is a video data application task, the determination of the task message corresponding to each of the one or more edge nodes according to the video processing task and the one or more edge nodes can include the following steps:

[0061] S410: splitting the video processing task according to the one or more edge nodes to obtain a center task and one or more edge tasks.

[0062] It can be understood that the video data application task refers to a task in which the processing logic is not fixed when video is used as a data source, such as a portrait search task, a license plate search task, a people flow statistics task, a vehicle flow statistics task, etc. in a specific time period or a specific place, and the video data of the required edge node can be selected according to the application requirements. If the video data is directly uploaded to the cloud center node, it will undoubtedly cause congestion of the uplink communication link and increase the load of the video processing cluster of the cloud center node. In the embodiments of the present application, for this type of task, the task can be split into a center task and an edge task. The center task and the edge task can both be local video processing tasks, or the center task can be a secondary processing task performed on the result obtained after the edge task is executed.

[0063] Specifically, after determining one or more edge nodes according to the video processing task, further, in combination with the time period of task execution, the type of executed task, the required video data source information, and the model data required by the task, etc., the video processing task is split. For example, the video processing task is to search for a target object in a specific area, determine which edge nodes are needed according to the area, according to whether the required video data is real-time video or historical video record at a specific time, and according to whether the model required by the task is a human feature model or a material feature model, the edge task is divided.

[0064] S430: match the one or more edge tasks with the one or more edge nodes, and determine the edge task corresponding to each edge node.

[0065] Optionally, the node information of the one or more edge nodes is obtained, which can include the physical address, network address, data processing type, function type, etc. of the edge node. Generally, the edge nodes can be grouped by tags, and each edge node is matched to the corresponding edge task according to the node information or tags of the edge node.

[0066] S450: generate a task message to be sent to each edge node according to each edge node and the edge task corresponding to each edge node.

[0067] It can be understood that the task message is a transmission signal containing edge task information, which can be sent to the corresponding edge node through the message access gateway.

[0068] Specifically, when the video processing task is a video viewing task, when the user needs to view the video, the edge node and the task message to be sent to the edge node are determined according to the relevant information of the video viewing task issued by the user, such as area information, video data type information, etc. The task message is a push stream instruction, which includes which edge node to obtain the video stream, whether the type of the video stream is the real-time video stream of the current camera or the historical video stream, the resolution of the video stream, and the start time and end time of the historical video stream, etc. When the video processing task is a video backup task, the user can configure backup rules, edge nodes requiring video backup, and backup video types in the video backup task. The cloud center node generates a task message according to the backup rules and edge nodes requiring video backup, etc. and sends the backup rules, backup types, etc. to the edge node.

[0069] S230: the cloud center node sends the task message to the video processing server of the edge node through the message access gateway.

[0070] In the embodiments of the present application, the access device of the cloud center node is additionally provided with a message access gateway in addition to the video access gateway, wherein the message access gateway and the video access gateway are both connected with the cloud center node and the edge node, and the message access gateway only transmits message data between the two, and the video stream data is transmitted through the video access gateway. Through the task message notification to each edge node, a part of the video processing logic can be sunk to the edge node, and the real-time performance of video processing is improved.

[0071] S250: The video processing server determines corresponding task data according to the task message, and reports the task data to the cloud center node.

[0072] In the embodiments of the present application, the edge node is provided with a video processing server, which can realize not only the communication with the center node, but also the localization of video processing.

[0073] In the embodiments of the present application, as shown in Figure 5 The step S250 can include the following steps.

[0074] S510: The video processing server obtains target video data according to the task message.

[0075] In the embodiments of the present application, the video processing server of the edge node obtains real-time video data from the camera or historical video from the local video storage according to the data type and other information specified in the task message, and if it is historical video, the start time and end time of the video can also be specified.

[0076] S530: The video processing server processes the target video data according to the task message to obtain task data.

[0077] In the embodiments of the present application, in the video data application task, the video processing server of the edge node can process the target video data according to the task type and model data required in the task message to obtain structured task data; in the video backup task, the target video data is compressed or cropped according to the backup rule in the task message to obtain effective task data that needs to be backed up.

[0078] S550: The video processing server reports the task data to the cloud center node.

[0079] Optionally, the type of the task data can be determined to determine the reported gateway, if the task data is structured data processed from the video data, the task data is reported to the cloud center node through the message access gateway, and if the task data is still video stream data, the task data is reported to the cloud center node through the video access gateway, which reduces the uplink bandwidth cost to a certain extent.

[0080] In the embodiments of the present application, as shown in Figure 6 When the video processing task is a video viewing task, the step S250 can further include the following steps:

[0081] S610: The video processing server acquires the target video stream of the edge node according to the task message.

[0082] In the embodiments of the present application, the video processing server of the edge node acquires the target video stream according to the push stream instruction (i.e. the task message), and the target video stream can be real-time video data or historical video data.

[0083] S630: The video processing server pushes the target video stream as task data to the cloud center node via the video access gateway of the cloud center node.

[0084] It can be understood that the video stream data is still transmitted to the cloud center node in the video viewing task, and therefore the video access gateway can be used for reporting. The edge node performs push streaming according to the task message, i.e. performs push streaming according to the user's demand, and there is no need to perform real-time push streaming of all edge nodes, thereby reducing the uplink bandwidth and the bandwidth cost.

[0085] S270: The cloud center node processes the video processing task according to the task data.

[0086] In the embodiments of the present application, the cloud center node and the edge node cooperatively complete the video processing task. The cloud center node further processes the task message reported by the edge node, which can be secondary processing of the data based on the task data, or can be display or storage of the task data.

[0087] In the embodiments of the present application, as shown in Figure 7 When the video processing task is a video data application task, according to the above steps S410-S430, the cloud center node processes the video processing task according to the task data, which can include the following steps:

[0088] S710: The video processing cluster of the cloud center node executes the center task in the video processing task according to the task data, and obtains the processing result of the video processing task.

[0089] In the embodiments of the present application, for the task data processed by the edge node according to the task message, the cloud center node can further use artificial intelligence or cloud computing and other related technologies to deeply mine and analyze the data, and obtain the final processing result.

[0090] S730: feeding back the processing result of the video processing task.

[0091] In the embodiments of the present application, when the video processing task is a video viewing task or a video backup task, according to the video processing task, the cloud center node transmits the task data to a display and control module of the cloud center node, and the video monitoring and control module writes the task data into a historical key video storage cluster, or the display and control module performs video display of the task data.

[0092] The cloud-edge collaborative video processing method provided in the embodiments of the present application realizes a cloud center node and edge node collaborative video processing scheme, improves the real-time performance of video processing, reduces the uplink bandwidth and the load of the center side video processing cluster and the center side historical video storage cluster, and can reduce the bandwidth cost, storage cost and cluster operation cost.

[0093] In this specification, the application scenario of video monitoring in the field of Internet of Things is taken as an example to provide exemplary embodiments. FIG. 8(1) is a cloud-edge architecture based on client / server for implementing video monitoring. As shown in FIG. 8(1), under this architecture, the center side (server side) includes a video monitoring display and control system / platform, a video processing cluster, a historical video storage cluster, and a same video access gateway; the edge side (client side) nodes each include a plurality of camera devices, a local gateway, and a historical video local storage device. After the camera collects video data, the edge node stores the video data to the historical video local storage device through the local gateway, and pushes the video data to the video access gateway of the center side through the local gateway; the center side (server side) receives the pushed video through the unified video access gateway, stores the video to the historical video storage cluster of the center side, and simultaneously pushes the video stream to the video processing cluster and the video monitoring display and control service in two paths. Under this architecture, all videos are pushed to the center side, and the center side and the edge side both store historical videos, with high storage redundancy. As the video data volume becomes larger, the bandwidth cost will increase sharply, the storage cost will also increase sharply, and the corresponding video processing cluster also needs more machines to process historical videos. This leads to a rapid increase in the cost of the entire video monitoring service with the increase of video data. FIG. 8(2) is a cloud-edge architecture diagram for the application scenario of video monitoring according to an embodiment of the present application. In this architecture, the local gateway of the edge side is changed to a function-enhanced video processing server, which enhances the computing power of the edge side to realize local processing of video data, and a message access gateway is added. The center side uses the gateway to issue task messages of video processing tasks, and the edge side implements local processing or on-demand pushing of videos according to the task messages issued by the message access gateway. For users, there is no difference in use compared with before, and they can still view and process real-time videos and historical videos from the center side.

[0094] The following describes the cloud-edge collaborative video processing method based on the architecture shown in FIG. 8(2) through the processing processes of the monitoring video data application task, the monitoring video viewing task, and the monitoring video backup task. It should be noted that the following processes are examples of the method provided by the present application, and are not the only limitation of the method provided by the present application.

[0095] Figure 9 FIG. 8(3) is a processing flow timing diagram when processing the monitoring video data application task on the cloud-edge collaborative architecture provided by the present application. As shown in FIG. 8(3), the process of the monitoring video data application task is as follows: Figure 9As shown in the video data application task processing scenario, after the user initiates the video data application task on the client operation interface or the interface of the display and control platform of the monitoring system, the task is sent to the video processing cluster on the center side, the video processing cluster splits the task into an edge side task and a center side task, the edge side task is sent to each edge node through the message access gateway, the video processing server of the edge node reads real-time video data from the camera or historical video data from the local storage according to the specific requirements of the edge task; according to the task type and model data specified in the edge task, the processing is started, and after the processing is completed, the edge side processing execution result is reported to the message access gateway, and the message access gateway is transmitted to the center side video processing cluster, the video processing cluster collects the processing execution results of each edge node, and then performs the center side task to process the data again, and finally returns the video processing result to the user.

[0096] In the processing scenario of the video data application task, the edge side performs preliminary processing on the local video (real-time video data or historical video data), improves the real-time performance of video processing, and reports the data to the center side, which is already processed structured data instead of video stream, reduces the uplink bandwidth and the load of the center node video processing cluster, and reduces the uplink bandwidth cost and the cluster operation cost.

[0097] Figure 10 The processing flow timing diagram when a monitoring video viewing task is processed on the cloud edge collaborative architecture provided by the application is as shown in Figure 10 As shown in the monitoring video viewing task processing scenario, the camera on the edge side is changed from the continuous push stream mode to the on-demand push stream mode, that is, only when the user triggers the viewing task, the video push stream is performed. When the user needs to view real-time or historical video, the center side video monitoring display and control system or module sends a push stream instruction to the video processing server of the specified edge node through the message access gateway, and the video processing server performs push stream according to the push stream type. If real-time video data is viewed, the video processing server directly transfers the camera video stream to the video gateway; if historical video data is viewed, the video processing server reads the local video storage and then pushes the stream to the video access gateway on the center side. The video access gateway transfers the video stream to the display interface of the video monitoring display and control system or module for the user to watch.

[0098] In the task scenario of monitoring video viewing, the edge side performs on-demand push stream according to the user demand, and no longer needs all edge nodes to push stream in real time, thereby reducing the uplink bandwidth and the bandwidth cost.

[0099] Figure 11 The processing flow timing diagram when a monitoring video backup task is processed on the cloud edge collaborative architecture provided by the application is as shown inFigure 11 As shown in the processing scenario of monitoring the video backup task, a user can configure a backup rule at the center side, and the video monitoring display and control function module sends the backup rule to the video processing server of the edge side node through the message access gateway. It can be understood that the configuration rule is included in the task message. The video processing server reads and processes the local stored history according to the configured backup rule, and then uploads the processed video that meets the configured backup rule to the center side through the video access gateway, and finally writes the video to the historical key video storage cluster at the center side.

[0100] In the task scenario of video backup, the edge side can crop the local historical video according to the configuration of the center side, and only push the video with effective information to the center side for backup. In addition, the video can be compressed before being pushed, which avoids real-time push streaming of full video and uploading of a large amount of static video data, that is, the key video is stored in multiple places, and the uplink bandwidth and the data amount of the historical video storage at the center side are reduced, thereby reducing the bandwidth and storage cost.

[0101] The embodiment of the application further provides a cloud-edge collaborative video processing device 1200, as shown in the figure, the device 1200 can include: Figure 12

[0102] The first task processing module 1210 is configured to acquire a video processing task by a cloud center node, and determine a task message to be sent according to the video processing task.

[0103] The message sending module 1220 is configured to send the task message to a video processing server of an edge node through a message access gateway by the cloud center node.

[0104] The second task processing module 1230 is configured to determine corresponding task data according to the task message by the video processing server, and report the task data to the cloud center node.

[0105] The third task processing module 1240 is configured to process the video processing task according to the task data by the cloud center node.

[0106] In the embodiment of the application, the first task processing module 1210 can include:

[0107] The node determination unit is configured to determine one or more edge nodes according to the video processing task.

[0108] The message generation unit is configured to determine a task message corresponding to the one or more edge nodes according to the video processing task and the one or more edge nodes respectively.

[0109] In the embodiment of the application, the message generation unit can further include: ​

[0110] splitting a video processing task into a center task and one or more edge tasks according to the one or more edge nodes;

[0111] task matching sub-unit, configured to match the one or more edge tasks with the one or more edge nodes, to determine the edge task corresponding to each edge node;

[0112] message generating sub-unit, configured to generate a task message to be sent to each edge node according to each edge node and the edge task corresponding to each edge node.

[0113] It should be noted that the apparatus provided in the above embodiments, in realizing its functions, only takes the above-mentioned division of each functional module as an example, and in actual application, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above-described functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0114] The embodiment of the present application provides a computer device, which comprises a processor and a memory, the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program is loaded and executed by the processor to realize a cloud edge collaborative video processing method provided by the above method embodiment.

[0115] Figure 13 A hardware structure schematic diagram of a device for implementing a cloud edge collaborative video processing method provided by the embodiment of the present application is shown, and the device can participate in constituting or containing the apparatus or system provided by the embodiment of the present application. As shown in the figure, Figure 13 The device 13 can include one or more (in the figure, 1302a, 1302b, …, 1302n are shown) processors 1302 (the processor 1302 can include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1304 for storing data, and a transmission device 1306 for communication function. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 13 The structure shown in the figure is only a schematic, which does not limit the structure of the above-mentioned electronic device. For example, the device 13 can also include more or less components than those shown in the figure, or have a structure different from that shown in the figure. Figure 13 Figure 13 ​different configurations are shown.

[0116] It should be noted that the one or more processors 1302 and / or other data processing circuitry described above can be referred to herein generally as "data processing circuitry". The data processing circuitry can be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuitry can be a single standalone processing module, or incorporated in whole or in part within any one of other elements of the device 13 (or mobile device). As referred to in embodiments of the present application, the data processing circuitry functions as a processor to control, for example, selection of variable resistance terminal paths connected to the interface.

[0117] The memory 1304 can be used to store software programs and modules of applications, and program instructions / data storage means corresponding to the method described in embodiments of the present application. The processor 1302 can execute various functions of applications and data processing by running the software programs and modules stored in the memory 1304, i.e. implement the cloud-edge collaborative video processing method described above. The memory 1304 can include a high-speed random access memory, and can further include a non-volatile memory such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 1304 can further include a memory remotely disposed relative to the processor 1302, which can be connected to the device 13 through a network. Examples of the network can include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0118] The transmission device 1306 is configured to receive or send data via a network. Examples of the network can include a wireless network provided by a communication provider of the device 13. In one example, the transmission device 1306 includes a network interface controller (NIC) which can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 1306 can be a radio frequency (RF) module configured to communicate with the Internet in a wireless manner.

[0119] The display can be, for example, a touch screen type liquid crystal display (LCD) which can enable a user to interact with a user interface of the device 13 (or mobile device).

[0120] The embodiment of the present application further provides a computer readable storage medium, which can be arranged in a server to save at least one instruction or at least one program for implementing a cloud-edge collaborative video processing method in the method embodiment, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the cloud-edge collaborative video processing method provided in the above method embodiment.

[0121] Optionally, in the embodiment, the storage medium can be located in at least one of a plurality of network servers of a computer network. Optionally, in the embodiment, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media capable of storing program codes.

[0122] The embodiment of the present application further provides a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the various optional embodiments.

[0123] It can be seen from the above embodiments of the cloud-edge collaborative video processing method, device, medium and equipment provided by the present application that,

[0124] The scheme provided by the present application enhances the data processing capability of the edge by replacing the edge video gateway with a video processing server at the edge; the efficiency of video data processing can be improved by adding a message access gateway at the cloud end to issue video processing tasks, and the cloud end further processes according to the processing result of the edge; and the cloud-edge collaboration reduces the load of the uplink bandwidth, the center-side video processing cluster and the center-side historical video storage cluster, and reduces the bandwidth cost, the storage cost and the cluster operation cost.

[0125] It should be noted that the above sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. The above describes the specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be executed in a different order from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or can be advantageous.

[0126] The various embodiments herein are described in progressive manner, and the same or similar parts among the various embodiments can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0127] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or can be instructed to complete by relevant hardware through a program. The program can be stored in a computer readable storage medium, and the storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk.

[0128] The above only describes the preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A cloud-edge collaborative video processing method, characterized in that, The method includes: The cloud center node acquires a video processing task and determines one or more edge nodes based on the video processing task; the video processing task is split according to the one or more edge nodes to obtain a central task and one or more edge tasks; the one or more edge tasks are matched with the one or more edge nodes to determine the edge task corresponding to each edge node; and a task message to be sent to each edge node is generated based on each edge node and the edge task corresponding to each edge node. The cloud center node sends the task message to the video processing server of the corresponding edge node through the message access gateway; The video processing server determines the corresponding task data based on the task message and reports the task data to the cloud center node; The cloud center node processes the video processing task based on the task data.

2. The method according to claim 1, characterized in that, The video processing server determines the corresponding task data based on the task message and reports the task data to the cloud center node, including: The video processing server obtains the target video data according to the task message; The video processing server processes the target video data according to the task message to obtain task data; The video processing server reports the task data to the cloud center node.

3. The method according to claim 1, characterized in that, The video processing server determines the corresponding task data based on the task message and reports the task data to the cloud center node, and also includes: The video processing server obtains the target video stream of the edge node according to the task message; The video processing server pushes the target video stream as task data to the cloud center node via the video access gateway of the cloud center node.

4. The method according to claim 1, characterized in that, The cloud center node processes the video processing task based on the task data, including: The video processing cluster of the cloud center node executes the central task in the video processing task according to the task data, and obtains the processing result of the video processing task; The processing results of the video processing task will be fed back.

5. The method according to claim 1, characterized in that, The cloud center node processes the video processing task based on the task data, and also includes: Based on the video processing task, the cloud center node displays or stores the task data.

6. A cloud-edge collaborative video processing device, characterized in that, The device includes: The first task processing module is used to: acquire video processing tasks from the cloud center node; determine one or more edge nodes based on the video processing tasks; split the video processing tasks based on the one or more edge nodes to obtain a central task and one or more edge tasks; match the one or more edge tasks with the one or more edge nodes to determine the edge task corresponding to each edge node; and generate a task message to be sent to each edge node based on each edge node and the edge task corresponding to each edge node. The message delivery module is used by the cloud center node to send the task message to the video processing server of the corresponding edge node through the message access gateway; The second task processing module is used by the video processing server to determine the corresponding task data according to the task message and report the task data to the cloud center node. The third task processing module is used by the cloud center node to process the video processing task based on the task data.

7. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement a cloud-edge collaborative video processing method as described in any one of claims 1 to 5.

8. A computer device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor to implement a cloud-edge collaborative video processing method as described in any one of claims 1 to 5.

9. A computer program product comprising computer instructions stored in a computer-readable storage medium; a processor of a computer device reading the computer instructions from the computer-readable storage medium, the processor executing the computer instructions to cause the computer device to perform a cloud-edge collaborative video processing method as described in any one of claims 1 to 5.

Images