Business management method, platform, service delivery system, and computer storage medium

Abstract

A service management method, platform, service delivery system, and computer storage medium. The service management method includes: obtaining monitoring information of edge nodes through a service delivery platform, and selecting available edge nodes from the obtained monitoring information; creating one or more transactions, each of the transactions including one or more artificial intelligence (AI) services; and sending the created transactions to the available edge nodes through the service delivery platform. The embodiments of the present disclosure implement a cloud-native edge node management method. The transaction sending method is efficient and convenient, and the transaction management mechanism is efficient and flexible. Through AI service orchestration, flexible construction and convenient controllability of transactions are ensured.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to, but are not limited to, the field of intelligent system technologies, and in particular, to a service management method, platform, and service delivery system. Background Art

[0002] Edge computing refers to using an edge device platform that integrates network, computing, storage, and application core capabilities on one side close to things or the data source to provide the nearest-end services nearby. Its application programs are initiated on the edge side, generating faster network service responses and meeting the basic needs of the industry in aspects such as real-time services, application intelligence, security, and privacy protection. Cloud computing can receive or access the historical data of edge computing in real time.

[0003] With the development of information technology, there are more and more artificial intelligence (AI) services deployed on edge devices, and the number of edge devices is increasing continuously. How to efficiently manage and deliver these services has become a key issue. Summary of the Invention

[0004] Embodiments of the present disclosure provide a service management method, device, and computer storage medium, which can improve the picture playback effect.

[0005] Embodiments of the present disclosure provide a service management method, including: obtaining monitoring information of edge nodes through a service delivery platform, and selecting available edge nodes from the obtained monitoring information; creating one or more transactions, each created transaction including one or more artificial intelligence (AI) services; and sending the created transactions to the available edge nodes through the service delivery platform.

[0006] In an exemplary embodiment, each created transaction further includes at least one of the following: an AI service orchestration script, an image acquisition device corresponding to the transaction, and an available edge node corresponding to the transaction.

[0007] In an exemplary embodiment, the method further includes: initializing the available edge nodes.

[0008] In an exemplary embodiment, the initializing the available edge nodes includes:

[0009] selecting an application, where the application is a control management program that defines the operation of the edge node;

[0010] obtaining an image list from the service delivery platform and selecting a basic container image from the image list;

[0011] According to the selected application and base container image, an application container image is built through the service delivery platform and stored in the container repository, and the service delivery platform is used to control the available edge nodes to download the application container image and build and start the application container.

[0012] In an exemplary embodiment, the application includes: receiving a dispatched transaction, dynamically loading a transaction, parsing a service orchestration transaction, and starting a thread to run a transaction.

[0013] In an exemplary embodiment, creating a transaction includes:

[0014] Selecting one or more AI services;

[0015] Generating an AI service orchestration script according to the selected one or more AI services;

[0016] Selecting one or more image acquisition devices;

[0017] Creating a transaction by combining the one or more AI services, the AI service orchestration script, the selected image acquisition devices, and the available edge node information.

[0018] In an exemplary embodiment, the method further includes:

[0019] Obtaining the transaction dispatch or update status of the available edge nodes through the service delivery platform.

[0020] In an exemplary embodiment, the method further includes:

[0021] Controlling the applications on the edge nodes through the service delivery platform, and the control includes at least one of the following: creating, starting, stopping, and updating.

[0022] In an exemplary embodiment, the method further includes:

[0023] Establishing an association relationship between the applications and transactions of the edge nodes, where the application is a control and management program that defines the operation of the edge nodes;

[0024] Mounting a unified shared storage on multiple edge nodes, and the shared storage is used to store the application and transaction configuration data of multiple edge nodes.

[0025] In an exemplary embodiment, when a failure occurs in the available edge node, the method further includes:

[0026] Detecting whether there is a first edge node, where the first edge node is an idle edge node. When the first edge node exists, migrating the applications and transactions of the failed edge node to the first edge node;

[0027] When the first edge node does not exist, detect whether there is a second edge node. The application version of the second edge node is the same as that of the failed edge node, and the second edge node has sufficient resources to receive the transactions of the failed edge node. When the second edge node exists, delete the association between the application and the transactions of the failed edge node, and migrate the transactions of the failed edge node to the second edge node;

[0028] When neither the first edge node nor the second edge node exists, generate an alarm message.

[0029] In an exemplary embodiment, the method further includes:

[0030] Obtain and display the AI service processing results of one or more of the available edge nodes.

[0031] In an exemplary embodiment, obtain the AI service processing results of one or more available edge nodes through any one or more of the following service channels: Message Queuing Telemetry Transport protocol, Remote Dictionary Service, Distributed Publish-Subscribe Messaging System, Network Socket.

[0032] In an exemplary embodiment, multiple transactions are run in parallel on one edge node, and each transaction is started as a thread.

[0033] An embodiment of the present disclosure also provides a service management platform, including a memory; and a processor coupled to the memory, the processor being configured to execute the steps of the service management method as described in any one of the above based on instructions stored in the memory.

[0034] An embodiment of the present disclosure also provides a service delivery system, including the service management platform as described above, further including a service delivery platform and one or more edge nodes, the service management platform, the service delivery platform, and the edge nodes are interconnected through a network;

[0035] The service delivery platform is configured to monitor and control the one or more edge nodes, and send the transactions created by the service management platform to the one or more edge nodes;

[0036] The edge node is configured to receive the transactions sent by the service delivery platform, process the transactions, and return the processing results to the service management platform

[0037] In an exemplary embodiment, the service delivery system further includes one or more image acquisition devices, and the edge node and the image acquisition device are interconnected through a network;

[0038] The edge node processes the transaction, including: obtaining the image or video data collected by one or more of the corresponding image acquisition devices according to the received transaction, and processing the image or video data.

[0039] In an exemplary embodiment, the edge node runs multiple transactions in parallel. The transactions include customer face registration transaction, customer identification transaction, in-store visit frequency analysis transaction, and stranger reminder transaction, where:

[0040] The customer face registration transaction includes face detection service, face scaling service, face key point detection service, face alignment service, face feature extraction service, and face feature storage service;

[0041] The customer identification transaction includes video decoding service, image scaling service, object detection service, object tracking service, face detection service, face scaling service, face quality evaluation service, face alignment service, face attribute judgment service, face feature extraction service, face retrieval service, and stranger registration service;

[0042] The in-store visit frequency analysis transaction includes record upload service and record statistics service;

[0043] The stranger reminder transaction includes stranger in-store situation statistics service and reminder service.

[0044] The embodiment of the present disclosure also provides a computer storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the service management method described in any one of the above.

[0045] The service management method, platform, and service delivery system of the embodiment of the present disclosure dispatch the service delivery platform through the service management platform for transaction distribution, container image construction, edge node monitoring, etc., realizing a cloud-native edge node management method. The transaction distribution method is efficient and convenient, the transaction management mechanism is efficient and flexible, and through AI service orchestration, the flexible construction and convenient control of transactions are ensured.

[0046] Other features and advantages of the present disclosure will be described in the following description, and some will be obvious from the description, or understood by implementing the present disclosure. Other advantages of the present disclosure can be realized and obtained through the solutions described in the description and the drawings. Description of the Drawings

[0047] The drawings are used to provide an understanding of the technical solutions of the present disclosure, and constitute a part of the description. They are used together with the embodiments of the present disclosure to explain the technical solutions of the present disclosure, and do not constitute a limitation to the technical solutions of the present disclosure.

[0048] Figure 1 It is a schematic flowchart of a service management method according to an exemplary embodiment of the present disclosure;

[0049] Figure 2 Schematic diagram of the process for initializing an edge node in an exemplary embodiment of the present disclosure;

[0050] Figure 3 Schematic diagram of the process for issuing / updating a transaction in an exemplary embodiment of the present disclosure;

[0051] Figure 4 Schematic diagram of the linkage of multiple AI services in a vehicle violation detection transaction in an exemplary embodiment of the present disclosure;

[0052] Figure 5a and Figure 5b Schematic diagram of the structure of two service delivery systems in an exemplary embodiment of the present disclosure;

[0053] Figure 6 Schematic diagram of an application scenario of a store visit frequency detection transaction in an exemplary embodiment of the present disclosure;

[0054] Figure 7 Schematic diagram of the customer identification process in a store visit frequency detection transaction in an exemplary embodiment of the present disclosure;

[0055] Figure 8 Schematic diagram of the customer store visit frequency detection process in a store visit frequency detection transaction in an exemplary embodiment of the present disclosure;

[0056] Figure 9 Schematic diagram of the structure of a business management platform in an exemplary embodiment of the present disclosure. Detailed implementation manners

[0057] To make the objectives, technical solutions, and advantages of the present disclosure clearer and more understandable, the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be noted that, without conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other arbitrarily.

[0058] Unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should have the ordinary meanings understood by those of ordinary skill in the art to which the present disclosure belongs. The "first", "second", and similar terms used in the embodiments of the present disclosure do not denote any order, quantity, or importance, but are only used to distinguish different components. The terms such as "including" or "comprising" are intended to indicate that the elements or items before this word cover the elements or items listed after this word and their equivalents, without excluding other elements or items.

[0059] As Figure 1 shown, the embodiments of the present disclosure provide a business management method, including the following steps:

[0060] Step 101: Obtain the monitoring information of the edge nodes through the service delivery platform, and select available edge nodes from the obtained monitoring information.

[0061] The service management method of the embodiments of the present disclosure is applied to a service management platform. The service management platform is the service initiator, and the edge nodes are the operation units of transactions. The service delivery platform controls the edge nodes through the cloud network. In some exemplary embodiments, the edge components (Edge part) are pre-installed on the edge nodes during delivery, making them Kubernetes nodes (Node). The core applications (APP) of the edge nodes can be distributed through Kubernetes.

[0062] Kubernetes (abbreviated as K8S) is a brand-new distributed architecture solution based on container technology and an open-source container cluster management system. A Kubernetes cluster generally includes a control (Master) node and multiple Node nodes. The Master node is the cluster control node of K8S. Each K8S cluster needs to have a Master node to be responsible for the management and control of the entire cluster. Basically, all control commands of K8S are sent to it, and it is responsible for the specific execution process. The Node nodes are the workload nodes in the K8S cluster. Each Node node will be assigned some workloads by the Master node. When a certain Node node fails, the workloads on it will be automatically transferred by the Master node to other Node nodes.

[0063] In some exemplary embodiments, the method further includes: initializing the available edge nodes.

[0064] In some exemplary embodiments, initializing the available edge nodes includes:

[0065] Select an application (APP), where the application is a control management program that defines the operation of the edge nodes;

[0066] Obtain the image list from the service delivery platform, and select the base container image from the image list;

[0067] According to the selected application and the base container image, build an application container image through the service delivery platform, store the built application container image in the container repository, and control the available edge nodes to download the application container image and build and start the application through the service delivery platform.

[0068] Such as Figure 2As shown in the figure, the business management platform selects the application of the current required version; the business management platform obtains the list of basic container images from the service delivery platform and selects the basic container image from the list of basic container images; the business management platform obtains the monitoring information of the edge nodes from the service delivery platform and selects the available edge nodes from the obtained monitoring information of the edge nodes; the business management platform constructs the application container image according to the selected application and the basic container image through the service delivery platform and stores the application container image in the container repository, and controls the available edge nodes to download the application container image through the service delivery platform and construct and start the application container.

[0069] In the embodiments of the present disclosure, the basic container image may be a Linux operating system such as Redhat or Ubuntu, and the basic container image may provide a basic environment for the application to run.

[0070] In some exemplary embodiments, the application may include: receiving a dispatched transaction, dynamically loading a transaction, parsing service orchestration, and starting a thread to run a transaction, etc.

[0071] In some exemplary embodiments, the application may further include: a video decoding transaction and a data stack management transaction, etc. When the edge node requires video decoding services, the application may include a video decoding transaction and a data stack management transaction, where the video decoding transaction is responsible for decoding the received video, and the data stack management transaction is responsible for storing the decoded image data. In some other exemplary embodiments, when the edge node directly receives image data through the network, the application may not include a video decoding transaction and a data stack management transaction.

[0072] In this embodiment, after initializing the available edge nodes, the available edge nodes start the above application, that is, they have the ability to receive dispatched transactions, dynamically load transactions, parse service orchestration, and start a thread to run a transaction, etc.

[0073] Step 102: Create one or more transactions, and each created transaction includes one or more AI services;

[0074] In some exemplary embodiments, each created transaction further includes at least one of the following: an AI service orchestration script, the image acquisition device corresponding to this transaction, and the available edge node corresponding to this transaction.

[0075] In some exemplary embodiments, creating a transaction includes:

[0076] Select one or more AI services;

[0077] Generate an AI service orchestration script according to the selected one or more AI services;

[0078] Select one or more image acquisition devices;

[0079] Create a transaction by combining the code base of one or more AI services, AI service orchestration scripts, selected image acquisition devices, and available edge node information.

[0080] like Figure 3 As shown, when a transaction is issued and started, the business management platform selects one or more AI services and generates an AI service orchestration script based on the selected one or more AI services; the business management platform selects one or more image acquisition devices; the business management platform obtains edge node information from the service delivery platform and selects available edge nodes from the obtained edge node information; the business management platform packages one or more AI service code libraries with the AI service orchestration script and sends them to available edge nodes; the business management platform obtains the transaction issuance or update status of the edge node through the service delivery platform. When the edge node receives the issued transaction, it downloads the transaction package, dynamically loads the AI library, parses the service orchestration script, creates the AI service call sequence, and then starts the transaction thread.

[0081] In some exemplary embodiments, an edge node runs multiple transactions in parallel, and each transaction is started as a thread.

[0082] With the development of AI technologies such as speech recognition and image recognition, more and more AI services are being put into use. However, a single AI algorithm model is often only targeted at a single problem. In some scenarios, a transaction often requires the linkage of multiple AI services, such as Figure 4 As shown in the figure, taking illegal parking monitoring as an example, if the vehicle image is input into the vehicle model recognition model, the vehicle model can be identified. If the vehicle model needs to be identified, different AI services need to be input. However, application scenarios often need to use data sources such as cameras to solve scenarios such as illegal parking monitoring. Monitor the parking situation of vehicles in public places, determine whether there is illegal parking in the core area, and further identify the license plate number, brand and model of the illegal vehicle. The vehicle is detected through the linkage between the vehicle model recognition algorithm service and the license plate recognition algorithm service, and the corresponding illegal penalties are implemented.

[0083] In some exemplary embodiments, the AI service orchestration process can be drawn through a web page or other visual methods, and a corresponding AI service orchestration script can be generated according to the flowchart to realize the linkage of multiple AI services.

[0084] Step 103: Send the created transaction to one or more available edge nodes corresponding to the transaction through the service delivery platform.

[0085] In some exemplary embodiments, the method further comprises:

[0086] Obtain transaction delivery or update status of available edge nodes through the service delivery platform.

[0087] As Figure 3 shown, after receiving the dispatched transaction, the edge node first downloads the transaction package, then dynamically loads the AI library, parses the service orchestration script and creates the AI service call sequence, and then starts the transaction thread. After the transaction starts successfully, the status of the edge node is updated.

[0088] In some exemplary embodiments, the method further includes:

[0089] Controlling the applications on the edge node through the service delivery platform, and the control includes at least one of the following: creating, starting, stopping, and updating, etc.

[0090] Exemplarily, the service delivery platform can be designed and developed based on K8S KubeEdge. The edge node installs Edgepart to manage and run APP containers. The service delivery platform manages the life cycle (creating, starting, stopping, updating, etc.) of APP containers through Edge part. In this embodiment, the node control depends on the control ability of K8S for containers, so it needs to be placed in the service delivery platform. If it is placed in the business management platform and the edge node needs to be directly controlled, it is necessary to customize its own control (creating, updating, starting, stopping) logic and protocol, which will increase the development workload.

[0091] In some exemplary embodiments, the method further includes:

[0092] Obtaining and presenting the AI service processing results of one or more available edge nodes.

[0093] In some exemplary embodiments, through any one or more of the following service channels: Message Queuing Telemetry Transport (MQTT), Remote Dictionary Server (Redis), distributed publish-subscribe messaging system (such as Kafka), WebSocket (a full-duplex communication protocol based on Transmission Control Protocol (TCP)), etc., obtain the AI service processing results of one or more available edge nodes.

[0094] The business management method provided by the embodiments of the present disclosure schedules the service delivery platform through the business management platform for transaction dispatch, container image construction, edge node monitoring, etc., realizing a cloud-native edge node management method. The transaction dispatch method is efficient and convenient, the transaction management mechanism is efficient and flexible, and through AI service orchestration, the flexible construction and convenient control of transactions are ensured.

[0095] The embodiments of the present disclosure also provide a service delivery system, which includes the business management platform as described above, and also includes a service delivery platform and one or more edge nodes. The business management platform, the service delivery platform and the edge nodes are interconnected through a network, and the edge nodes and the image acquisition devices are interconnected through a network.

[0096] The service delivery platform is configured to monitor and control one or more edge nodes, and send the transactions created by the business management platform to one or more edge nodes;

[0097] The edge nodes are configured to receive the transactions sent by the service delivery platform, process the transactions, and return the processing results to the business management platform.

[0098] In some exemplary embodiments, the service delivery system further includes one or more image acquisition devices, and the edge nodes and the image acquisition devices are interconnected through a network;

[0099] The edge nodes process the transactions, including: obtaining the image or video data collected by the corresponding one or more image acquisition devices according to the received transactions, and processing the image or video data.

[0100] As Figure 5a and Figure 5b shown, the service delivery system provided by the embodiments of the present disclosure includes at least the following components: a business management platform, a service delivery platform, and edge nodes.

[0101] (1) Business management platform

[0102] The business management platform is the service initiation end, and its functional modules include but are not limited to an edge node management module, a camera management module, a transaction management module, an APP management module, an AI service management module, an AI service display module, etc.

[0103] Edge node management module: responsible for configuring all edge node information of the business management platform, and interacting with the edge monitoring module of the service interaction platform to obtain the resource usage of the current edge nodes.

[0104] Camera management module: responsible for recording the configuration information of all camera video streams (exemplarily, the configuration information includes information such as stream address, camera model, camera location, camera manufacturer, etc.), and the association relationship between each camera and the transaction (that is, the data source of the transaction comes from which camera or which several cameras).

[0105] Transaction Management Module: Define an AI service capability as a transaction. When creating a transaction, it is necessary to configure the AI service, generate an AI service orchestration script, select a camera, select available edge nodes, etc. In Figure 5, a transaction is defined using the in-store frequency or departure detection as an example. This transaction needs to interact with other modules to obtain necessary configuration information.

[0106] APP Management Module: APP refers to the control and management program that defines the operation of edge nodes. In the embodiments of the present disclosure, the main functions of the APP are video stream decoding, AI service management and scheduling, etc. APP management includes starting the APP, stopping the APP, updating the APP, etc.

[0107] AI Service Management Module: This module is responsible for packaging the AI service into a module that can be scheduled by the APP, and at the same time defining the linkage and orchestration relationship between multiple AI services. In the embodiments of the present disclosure, specifically how to define the linkage and orchestration between AI services needs to be defined according to the actual usage scenario, and the present disclosure does not limit this.

[0108] AI Service Display Module: Responsible for displaying the AI service processing results from the edge nodes.

[0109] (2) Service Delivery Platform

[0110] The service delivery platform can be designed and developed based on K8S KubeEdge. The Edge part is installed on the edge side to manage the edge containers running the APP. The service delivery platform manages the life cycle (creation, start, stop, deletion, etc.) of the APP containers through the Edge part. At the same time, through the Open API of the service delivery platform, the business management platform schedules the service delivery platform to perform APP container image construction, transaction update, edge node control, and return the edge node monitoring information to the business management platform.

[0111] KubeEdge is a native edge computing platform for Kubernetes. The KubeEdge architecture includes two parts, namely the cloud side and the edge side. The cloud side is responsible for the distribution of applications and configurations, and the edge side is responsible for running edge applications and managing access devices.

[0112] (3) Edge Node

[0113] The edge node is the operating unit of the transaction. The Edge part is pre-installed on the edge node during delivery, making it a K8S Node node. The core application APP of the edge node is distributed by K8S.

[0114] APP applications mainly include video decoding transactions, data stack management transactions, receiving distribution transactions, dynamic loading transactions, parsing service orchestration transactions, starting thread running transactions, etc.

[0115] An edge node can run multiple transactions in parallel. Each transaction can be started as a thread, and different transactions can be executed in parallel.

[0116] In some exemplary embodiments, an edge node runs two transactions: Transaction 1 and Transaction 2. Exemplarily, Transaction 1 can be used for face detection; Exemplarily, Transaction 2 can be used for human body detection. Since the requirements for the shooting accuracy and shooting range of the camera may be different for Transaction 1 and Transaction 2, separate detection is beneficial for controlling the shooting accuracy and shooting range of the corresponding camera respectively. However, when returning the results, the detection results of Transaction 1 and Transaction 2 for the same detection target can be packaged and sent back to the business management platform.

[0117] The results generated by the transactions are uniformly reported by the edge node to the business management platform through any one or more of the following service channels: Message Queuing Telemetry Transport Protocol, Remote Dictionary Service, Distributed Publish-Subscribe Messaging System, Network Socket, etc.

[0118] In some exemplary embodiments, the results output by the edge AI transactions can include: information such as display, alarm, notification, etc.

[0119] Taking the off-duty detection of this solution as an example, when the AI service processing result of the edge node finds that there is no relevant person in the current position, it sends information such as (a certain position: off-duty) to the AI service display module of the business management platform. The AI service display module can pop up an alarm box "someone is off-duty at a certain position" and send an alarm to the relevant management personnel. At the same time, it can pull the processed picture of the AI service through the WebSocket protocol or other methods to display the information of the absent person and the position information.

[0120] Taking the in-store frequency detection of this solution as an example, when a certain customer appears for the first time on the same day, the AI service processing result returns information such as the customer ID and in-store time to the AI service display module. After receiving the information of the customer, the AI service display module first records the customer's current in-store information in the database, and then counts the in-store times of the customer within a specified time period (such as 1 year). It can pop up a prompt message on the web (Web) side that the in-store times in 1 year > the preset in-store times threshold (such as 5 times) so that the store clerk can focus on this customer. In actual use, the size of the preset in-store times threshold can be adjusted according to needs. For example, when it is necessary to pay attention to strange customers, the information of the customers who come to the store for the first time can be popped up, the in-store times = 1, and the size of the preset in-store times threshold can be configured through the AI service display module.

[0121] When the service delivery platform monitors that a certain edge node (such as edge node 1) fails (such as a downtime), it reports the fault information to the edge node management module of the service management platform for processing. Since in the embodiments of the present disclosure, the available edge nodes run application (APP) containers and the issued transactions simultaneously, the high availability can be achieved in the following two cases:

[0122] 1) If there is an idle edge node (such as edge node 2), since the application is packaged into an application container (docker) image, the application can be migrated to other available idle edge nodes (such as edge node 2) through the high availability migration ability of K8S. However, the transaction data cannot be directly migrated from the failed edge node 1 to the idle edge node 2. Therefore, in some exemplary embodiments, in the transaction management module of the service management platform, the application and the transaction of each edge node can be associated, and a unified shared storage can be mounted on each edge node. The shared storage is used to store the application transaction configuration data of each edge node. Therefore, when migrating the application and transaction data of edge node 1 to edge node 2, only the application transaction configuration data of edge node 1 needs to be read when starting the application on edge node 2.

[0123] Exemplarily, the association relationship between the application and the transaction of the edge node can be expressed as APP-ID1 (transaction 1, transaction 2), where ID1 of APP-ID1 can be the name of the docker image into which the application is packaged, and the application and the transaction are associated through this ID.

[0124] If there is no idle edge node, but there is an edge node with the same application version as the failed edge node, and the resource usage of this edge node meets the requirement of accepting more transactions, then on the service management platform, through the edge node management module, the association relationship between the application and the transaction of the failed edge node in the transaction management module is deleted. For example, the record information of transactions 1 and 2 running on edge node 1 and the association relationship with APP-ID1 are deleted, and the association relationship between APP-ID1 and transactions 1 and 2 is deleted on the corresponding shared storage, so as to avoid multiple runs of a transaction when edge node 1 is repaired. The corresponding transactions 1 and 2 are issued to other available edge nodes with the same version (such as edge node 2) to run through the transaction management module, and the association relationship between the edge node application and the transaction in the shared storage is updated, such as configuring APP-ID2 (transaction 4, transaction 1, transaction 2), and the association relationship between the edge node application and the transaction in the transaction management module is updated, such as recording APP-ID2 (transaction 4, transaction 1, transaction 2).

[0125] 2) If there is neither an idle edge node nor an edge node running the same version of APP that meets the requirements, the edge node management module triggers an alarm mechanism to trigger manual intervention processing by sending text messages, WeChat messages, emails, etc.

[0126] When a certain camera fails, when the application of the edge node performs video encoding and decoding, it is found that the program cannot run normally, and an alarm message is sent. After the edge monitoring module of the service delivery platform receives the information, the edge monitoring module feeds back the received information to the edge node management module of the service management platform, triggering an alarm mechanism, and triggering manual intervention processing by sending text messages, WeChat, emails, etc. In this embodiment, the edge monitoring module is placed on the service delivery platform because the monitoring system of K8S such as Prometheus (an open-source system monitoring and alarm system) can be directly used. The service management platform focuses more on business capability management, while the service delivery platform focuses on delivery operation and maintenance management.

[0127] When a certain transaction needs to optimize the AI service, the business management method of the present disclosure can complete the transaction hot update. Taking the update of transaction 2 as an example, first, through the transaction management module of the service management platform, call the transaction update module of the service delivery platform to notify the application APP-ID1 of edge node 1 to delete transaction 2. After receiving the deletion command, APP-ID1 stops and exits the transaction 2 thread through the main thread, and deletes the edge transaction configuration data APP-ID1 (transaction 1, transaction 2) on the centralized shared storage, which becomes APP-ID1 (transaction 1). This edge node reports to the transaction update module of the service delivery platform that this deletion task has been completed. Then, the transaction management module of the service management platform regenerates the transaction and issues it. The edge node to be issued does not need to be selected and is still edge node 1. Start the transaction issuing and transaction starting processes, and update the association relationship between the application of edge node 1 and the transaction in the transaction management module, as well as the transaction configuration data APP-ID1 (transaction 1, transaction 2) of edge node 1 in the shared storage.

[0128] The following takes the in-store frequency detection transaction as an example to illustrate the business management method of the embodiments of the present disclosure.

[0129] In business operations such as commerce and stores, it is often necessary to master the situation of customers arriving at the store, including daily, weekly, monthly, etc. in-store statistics, to draw a portrait of customers and adjust business strategies. Previously, data was often obtained by manual recording of the arrival of target customers by store clerks, but this method is inefficient and the data acquisition is inaccurate, and it is easy to miss reports. In response to this situation, the embodiments of the present disclosure design a customer in-store frequency statistics system based on edge computing, which can provide a safe, efficient, and intelligent customer in-store statistics method. The system includes a camera, a network, and edge devices (i.e., edge nodes) in terms of hardware, and includes a video stream processing module, a target detection module, a target tracking module, a face correction module, a face recognition module, etc., as well as functional modules such as data storage, analysis and calculation, display, and synchronization with the cloud in terms of software. The deployment architecture is as Figure 6 shown.

[0130] In the embodiments of the present disclosure, by deploying edge devices inside or near a store, the edge devices are close to the application scenario, enabling low-latency video transmission and highly real-time visible customer in-store records or reminders.

[0131] This system consists of two parts: software and hardware.

[0132] The hardware includes: image acquisition devices (such as cameras), image processing devices (such as edge nodes), network devices capable of connecting image acquisition devices and image processing devices, a service delivery platform, and a business management platform.

[0133] The software includes: The software of this system mainly runs on the edge node, including a video decoding module, an object detection module, an object tracking module, a face correction module, a face recognition module, etc. What this disclosure calls a "module" can also be called a "service". Based on the processing results of each module, the in-store situation of customers is stored, and in-store frequency analysis and display are performed daily, weekly, monthly, annually, etc. as needed.

[0134] The object detection module is configured to find objects from a scene (picture), including two processes: detection (where) and recognition (what).

[0135] The object tracking module is configured to establish the positional relationship of the object to be tracked in a continuous video sequence and obtain the complete motion trajectory of the object. The object tracking module calculates the exact position of the object in the next frame of the image based on the target coordinate position of the first frame of the given image. During the movement, the object may present some image changes, such as changes in posture or shape, changes in scale, background occlusion, or changes in light brightness, etc. Object tracking technology is one of the hotspots in the field of computer vision research and has been widely used. Tracking and focusing of cameras, automatic target tracking of drones, etc. all require the use of object tracking technology. In addition, there is also the tracking of specific objects, such as human tracking, vehicle tracking in traffic monitoring systems, face tracking, and gesture tracking in intelligent interaction systems.

[0136] The face correction module is configured to, when the detected face angle is not very correct, align it through face key point detection and methods such as rotation transformation based on key points.

[0137] The face recognition module includes a face recognition sub-module and a face verification sub-module. The face recognition sub-module is configured to classify a face into a specific identification; the face verification sub-module is configured to determine whether a pair of pictures belong to the same person (Verification).

[0138] In some exemplary embodiments, multiple transactions can run in parallel on an edge node. Exemplarily, the multiple transactions include customer face registration transaction, customer identification transaction, in-store frequency analysis transaction, and stranger reminder transaction, where:

[0139] The customer face registration transaction includes face detection service, face scaling service, face key point detection service, face alignment service, face feature extraction service, and face feature storage service;

[0140] The customer identification transaction includes video decoding service, image scaling service, object detection service, object tracking service, face detection service, face scaling service, face quality evaluation service, face alignment service, face attribute judgment service, face feature extraction service, face retrieval service, and stranger registration service;

[0141] The in-store frequency analysis transaction includes record upload service and record statistics service;

[0142] The stranger reminder transaction includes stranger in-store situation statistics service and reminder service.

[0143] In some exemplary embodiments, the system of the present disclosure includes but is not limited to the following functions:

[0144] (1) Customer face registration.

[0145] As Figure 7 shown, in order to identify the customer's identity, the system of the present disclosure needs to first complete face registration based on the customer's picture. The management personnel can upload the picture through the interface provided by the edge node, such as a web service. After the picture is uploaded, the customer registration function is triggered. First, the picture is sent to the target detection module (this module includes a face detection algorithm) to identify the customer's face information.

[0146] In some cases, the face may be tilted or otherwise, which may affect the accuracy of customer identification. At this time, it is necessary to align the detected face through the face correction module. The alignment process includes first scaling the face size, calling the face key point detection algorithm to identify the face key point information, and passing it into the face alignment algorithm for face alignment to achieve head tilt correction. Then, call the face recognition module to extract the face features. Finally, call the face storage module to store the extracted features in the edge node. To protect the customer's privacy, the customer image can be deleted after the face features are extracted. In the embodiments of the present disclosure, by storing user data in the edge device and only storing the customer's face features, the security and privacy of customer data are guaranteed.

[0147] (2) Customer identification, in-store frequency analysis

[0148] The embodiments of the present disclosure take 3 cameras as an example for collection to avoid the influence of a single camera on the extraction of user face data due to occlusion or other reasons. As Figure 8As shown in the figure, first, the input video stream is decoded by the video decoding module, and the decoded image is scaled. Then, the target detection module is called to detect the pedestrian information in the image, and the target tracking module is used to determine whether the pedestrian appears for the first time.

[0149] If the target tracking module determines that the pedestrian appears for the first time, face detection is performed on the pedestrian. After the face is detected, the image is scaled, and the input face quality evaluation module is used to determine whether the face is blurred or occluded. If the face quality is unqualified, the process returns to the next round of video stream processing; if the face quality is qualified, face alignment is performed to correct head tilts and other situations. The corrected result is input into the face attribute module to obtain auxiliary information such as age and gender. Then, the face feature extraction module is called to extract face features, and the extracted features are used to retrieve the matching customer in the face feature library. After comprehensive matching with the face attribute features, the final customer is determined, and the pedestrian is bound to the customer, and the customer's arrival time at the store and the customer's trajectory information are recorded.

[0150] If the target tracking module determines that the pedestrian does not appear for the first time, it is determined whether the pedestrian is bound to a customer. If bound, the customer's arrival time at the store and the customer's trajectory information are recorded; otherwise, as a stranger, the customer's arrival time at the store is recorded, and then the process returns to the next round of video stream processing.

[0151] If it is found after target detection that the pedestrian being tracked has disappeared, the customer's departure time from the store is recorded. Through technologies such as multiple cameras, target tracking, and accurate face recognition in the embodiments of the present disclosure, the real-time information of customers arriving at the store can be more accurately captured, and the arrival and departure situations of customers are recorded, providing a reference for operators.

[0152] The customer arrival records and in-store trajectory information can be stored on the edge node, which can ensure the security of sensitive data. However, problems such as device damage may cause data loss. Therefore, under the condition of ensuring data security (such as encrypted transmission), the customer arrival records and in-store trajectory information can be uploaded to the cloud.

[0153] When the operator needs to analyze customer behavior, the customer arrival and departure timestamps stored in a database or other storage methods are extracted, timestamp calculations are performed, and statistics such as the number of times of arriving at the store per day, per week, and per month are generated. Detecting one arrival record and one departure record on a certain day is regarded as one arrival at the store. In addition, while recording the arrival record information, the present disclosure also records the customer's in-store trajectory information, and the customer's trajectory in the store can be analyzed as needed, providing a reference for customer portraits for some operators.

[0154] (3) Statistics and reminder for strangers arriving at the store

[0155] Such as Figure 8As shown, for registered customers, complete customer in-store visit frequency statistics and activity track analysis can be achieved. In many cases, there are some customers who are not registered but also frequently visit the store, and the store staff needs to be reminded to pay attention and guide the users to register to enhance customer stickiness.

[0156] In the present disclosure, for strangers, a stranger face database is designed. When it is retrieved that the target pedestrian is not a registered customer, the stranger face database is searched. If it is an unregistered stranger, the stranger is first registered and then the in-store time and out-store time are recorded. After the recording is completed, the statistics of the stranger's in-store situation is triggered. If the number of in-store visits > the threshold, the location and avatar of the pedestrian are sent to the store staff for reminder and guidance to register.

[0157] In the embodiments of the present disclosure, by introducing a stranger face database, multiple in-store reminders for strangers are realized, guiding the store staff to pay attention to potential customers and improving the business operation effect.

[0158] (4) Display of analysis results

[0159] In the embodiments of the present disclosure, a Web application can be deployed on the edge node. Through the edge application service, it communicates with the front end (such as the customer in-store business system deployed at the edge) to obtain user in-store records. The front end can view the user records through a browser, or view the in-store records of specified category users.

[0160] In the embodiments of the present disclosure, a Web application can also be deployed in the cloud. Through the cloud application service, it communicates with the customer in-store business system deployed at the edge to obtain the in-store records, and the cloud Web page is responsible for the display.

[0161] The embodiments of the present disclosure also provide a business management platform, including a memory; and a processor coupled to the memory, the processor being configured to execute the steps of the business management method as described in any of the previous items based on the instructions stored in the memory.

[0162] As Figure 9 As shown, in an example, the business management platform may include: a processor 910, a memory 920, a bus system 930, and a transceiver 940. Among them, the processor 910, the memory 920, and the transceiver 940 are connected through the bus system 930. The memory 920 is used to store instructions, and the processor 910 is used to execute the instructions stored in the memory 920 to control the transceiver 940 to send signals. Specifically, the transceiver 940 can obtain the monitoring information of the edge node through the service delivery platform under the control of the processor 910. The processor 910 creates one or more transactions according to the obtained monitoring information of the edge node. The transactions include one or more AI services, AI service orchestration scripts, image acquisition devices, and available edge nodes. The transceiver 940 issues the created transactions to one or more available edge nodes through the service delivery platform under the control of the processor 910.

[0163] It should be understood that the processor 910 may be a central processing unit (CPU), and the processor 910 may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0164] The memory 920 may include a read-only memory and a random access memory, and provide instructions and data to the processor 910. A part of the memory 920 may also include a non-volatile random access memory. For example, the memory 920 may also store information about the device type.

[0165] In addition to including a data bus, the bus system 930 may also include a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, in Figure 9 all kinds of buses are labeled as the bus system 930.

[0166] In the implementation process, the processing executed by the processing device may be completed by the integrated logic circuit in the hardware of the processor 910 or instructions in the form of software. That is, the method steps of the embodiments of the present disclosure may be embodied as being executed and completed by the hardware processor, or by a combination of the hardware and software modules in the processor. The software module may be located in a storage medium such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 920, and the processor 910 reads the information in the memory 920 and combines its hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0167] Embodiments of the present disclosure also provide a computer storage medium. The computer storage medium stores executable instructions. When the executable instructions are executed by a processor, the business management method provided in any of the above embodiments of the present disclosure can be implemented. The business management method can obtain monitoring information of edge nodes through a service delivery platform; create one or more transactions according to the obtained monitoring information of the edge nodes, where the transactions include one or more AI services, AI service orchestration scripts, image acquisition devices, and available edge nodes; and send the created transactions to one or more available edge nodes through the service delivery platform, thereby implementing a cloud-native edge node management method. The transaction sending method is efficient and convenient, and the transaction management mechanism is efficient and flexible. Through AI service orchestration, flexible construction and convenient control of transactions are ensured. The method of driving the business management of the business management platform by executing the executable instructions is basically the same as the business management method provided in the above embodiments of the present disclosure, and will not be elaborated here.

[0168] The business management method, platform, service delivery system, and computer storage medium provided in the embodiments of the present disclosure realize a cloud-native edge node management method by scheduling the service delivery platform through the business management platform for transaction sending, container image construction, edge node monitoring, etc. The transaction sending method is efficient and convenient, and the transaction management mechanism is efficient and flexible. Through AI service orchestration, flexible construction and convenient control of transactions are ensured.

[0169] Those of ordinary skill in the art will understand that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, and their appropriate combinations. In the hardware implementation, the division of functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component can have multiple functions, or a function or step can be executed by several physical components in cooperation. Some or all components can be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, a communication medium typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium.

[0170] Although the embodiments disclosed in this disclosure are as above, the content described is only an embodiment adopted for the convenience of understanding this disclosure, and is not intended to limit this disclosure. Any person skilled in the art within the scope of this disclosure can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in this disclosure. However, the protection scope of this disclosure shall still be subject to the scope defined by the appended claims.

Claims

1. A service management method, characterized in that, Including: Obtain the monitoring information of edge nodes through the service delivery platform, and select available edge nodes from the obtained monitoring information; Create one or more transactions, and each created transaction includes one or more artificial intelligence (AI) services; Send the created transactions to the available edge nodes through the service delivery platform; Establish the association relationship between the applications and transactions of the edge nodes, where the application is a control and management program that defines the operation of the edge nodes; Mount a unified shared storage on multiple edge nodes, and the shared storage is used to store the application and transaction configuration data of multiple edge nodes; When a failure occurs in the available edge nodes, the method further includes: Detect whether there is a first edge node, where the first edge node is an idle edge node. When the first edge node exists, migrate the applications and transactions of the failed edge node to the first edge node; When the first edge node does not exist, detect whether there is a second edge node, where the application version of the second edge node is the same as that of the failed edge node and the second edge node has sufficient resources to receive the transactions of the failed edge node. When the second edge node exists, delete the association relationship between the applications and transactions of the failed edge node, and migrate the transactions of the failed edge node to the second edge node; When neither the first edge node nor the second edge node exists, generate an alarm message.

2. The service management method according to claim 1, wherein Each created transaction further includes at least one of the following: an AI service orchestration script, the image acquisition device corresponding to the transaction, and the available edge node corresponding to the transaction.

3. The service management method according to claim 1, characterized in that The method further includes: initializing the available edge nodes.

4. The service management method according to claim 3, characterized in that The initializing of the available edge nodes includes: Select an application, where the application is a control and management program that defines the operation of the edge nodes; Obtain a mirror list from the service delivery platform, and select a base container mirror from the mirror list; According to the selected application and base container mirror, build an application container mirror through the service delivery platform and store the application container mirror in the container repository. Control the available edge nodes to download the application container mirror through the service delivery platform and build and start the application container.

5. The service management method according to claim 4, characterized in that The application includes: receiving a sent transaction, dynamically loading a transaction, parsing a service orchestration transaction, and starting a thread to run a transaction.

6. The service management method according to claim 1, wherein The creating of one transaction includes: Select one or more AI services; Generate an AI service orchestration script according to the selected one or more AI services; Select one or more image acquisition devices; Create the one or more AI services, the AI service orchestration script, the selected image acquisition devices, and the available edge node information as one transaction.

7. The service management method according to claim 1, characterized in that The method further includes: Obtain the transaction sending or update status of the available edge nodes through the service delivery platform.

8. The service management method according to claim 1, characterized in that The method further includes: Control the applications on the edge nodes through the service delivery platform, and the control includes at least one of the following: creating, starting, stopping, and updating.

9. The service management method according to claim 1, wherein The method further includes: Obtain the AI service processing results of one or more of the available edge nodes and display them.

10. The service management method according to claim 9, characterized in that Obtain the AI service processing results of the one or more available edge nodes through any one or more of the following service channels: Message Queuing Telemetry Transport Protocol, Remote Dictionary Service, Distributed Publish-Subscribe Messaging System, Network Socket.

11. The service management method according to claim 1, characterized in that A plurality of transactions are run in parallel on one of the edge nodes, and each of the transactions is started as a thread.

12. A service management platform, characterized in that, It includes a memory; and a processor coupled to the memory, and the processor is configured to execute the steps of the service management method according to any one of claims 1 to 11 based on instructions stored in the memory.

13. A service delivery system, characterized in that, It includes a service management platform according to claim 12, and further includes a service delivery platform and one or more edge nodes, and the service management platform, the service delivery platform and the edge nodes are interconnected through a network; The service delivery platform is configured to monitor and control the one or more edge nodes, and send the transactions created by the service management platform to the one or more edge nodes; The edge node is configured to receive the transactions sent by the service delivery platform, process the transactions, and return the processing results to the service management platform.

14. The service delivery system according to claim 13, wherein It further includes one or more image acquisition devices, and the edge node and the image acquisition device are interconnected through a network; The edge node processes the transaction, including: obtaining the image or video data collected by the corresponding one or more of the image acquisition devices according to the received transaction, and processing the image or video data.

15. The service delivery system according to claim 13, characterized in that, A plurality of transactions are run in parallel on the edge node, and the transactions include customer face registration transactions, customer identification transactions, in-store visit frequency analysis transactions, and stranger reminder transactions, where: The customer face registration transaction includes face detection service, face scaling service, face key point detection service, face alignment service, face feature extraction service, and face feature storage service; The customer identification transaction includes video decoding service, image scaling service, target detection service, target tracking service, face detection service, face scaling service, face quality evaluation service, face alignment service, face attribute judgment service, face feature extraction service, face retrieval service, and stranger registration service; The in-store visit frequency analysis transaction includes record upload service and record statistics service; The stranger reminder transaction includes stranger in-store situation statistics service and reminder service.

16. A computer storage medium, characterized in that, A computer program is stored thereon, and when the program is executed by the processor, it implements the service management method according to any one of claims 1 to 11.

Images