An algorithm usage method, ecn, client and algorithm usage system

Abstract

The embodiment of the application discloses an algorithm using method, an ECN, a client and an algorithm using system, the method is applied to an edge computing node ECN side, the ECN includes: a device application layer; the method includes: receiving an algorithm program package through the device application layer, and analyzing the algorithm program package to obtain an algorithm application APP contained in the algorithm program package; the algorithm program package is provided by an algorithm manufacturer according to a preset uniform standard, and is imported into the ECN through a client calling a standard software development kit SDK interface; the obtained video frame data is analyzed by the algorithm application APP.Through the embodiment scheme, the algorithm manufacturer does not need to modify the existing algorithm interface, and can directly import the algorithm program package when connecting different algorithms, without secondary development.

Description

Technical Field

[0001] This article discusses algorithm application technologies, particularly an algorithm usage method, ECN, client, and algorithm usage system. Background Technology

[0002] In recent years, surveillance systems have evolved from the traditional analog era to the digital era and now to the era of ubiquitous intelligence. Intelligent analysis algorithms, such as facial recognition algorithms, have been widely applied in surveillance systems. Today, more and more manufacturers are developing intelligent analysis algorithms, but these algorithms require suitable platforms. Surveillance systems, as systems inherently capable of capturing various scenarios, combined with intelligent analysis algorithms, have achieved intelligent monitoring in diverse environments.

[0003] Currently, there is no unified standard for the external interfaces of intelligent analysis algorithms from different vendors. Integrating an intelligent analysis algorithm into a monitoring system requires understanding the vendor's SDK (Software Development Kit) and developing accordingly. Furthermore, different vendors' intelligent analysis algorithms necessitate redeveloping and replacing the software, making it impossible to quickly replace different vendors' intelligent analysis algorithms with a single device, thus increasing the development cost of the monitoring system.

[0004] The current solution involves algorithm vendors or associations developing unified algorithm protocol standards, with vendors then implementing interfaces according to these standards. Finally, monitoring vendors develop monitoring devices that conform to the protocol standards and interface with the intelligent analysis algorithms provided by the algorithm vendors. However, existing technologies require algorithm vendors to actively cooperate in implementing a unified standard protocol, but this unified standard cannot account for the differences between various algorithm vendors. The interfaces provided by the protocol cannot meet the needs of all algorithm vendors, and since the protocol is determined by the protocol-setting party, the timeliness and accuracy of protocol updates cannot be guaranteed. Summary of the Invention

[0005] This application provides an algorithm usage method and system that allows algorithm vendors to directly import algorithm packages without modifying existing algorithm interfaces and to integrate different algorithms without requiring secondary development.

[0006] This application provides an algorithm usage method applied to the edge computing node ECN side, wherein the ECN may include: a device application layer; the method may include:

[0007] The device application layer receives the algorithm package and parses it to obtain the algorithm application APP contained in the algorithm package. The algorithm package is provided by the algorithm vendor according to a preset unified standard and is connected to the ECN on the client. It is imported into the ECN by calling the standard software development kit (SDK) interface on the client.

[0008] The algorithm is applied to the APP to perform algorithmic analysis on the acquired video frame data.

[0009] In an exemplary embodiment of this application, before the algorithm package is imported into the ECN via the client calling the standard software development kit (SDK) interface, the method may further include:

[0010] ECN equipment manufacturers provide algorithm vendors with a standard SDK interface and a standard method for packaging the algorithm packages, so that the algorithm vendors can package their respective algorithms according to the standard method, obtain the corresponding algorithm packages, and provide the algorithm packages through the standard SDK interface.

[0011] In an exemplary embodiment of this application, the ECN may further include: a device driver; before performing algorithmic analysis on the acquired video frame data through the algorithm application APP, the method further includes:

[0012] The device application layer initiates streaming based on the video stream's Uniform Resource Locator (URL) address and sends the obtained stream data to the device driver; the video stream URL address is input through the client's input window.

[0013] The device application layer communicates with the client, and based on the communication result, the device driver decodes the bitstream data according to the decoding service issued by the client, and obtains the algorithm analysis instruction sent by the client to instruct the algorithm application APP to perform algorithm analysis.

[0014] In an exemplary embodiment of this application, the step of communicating with the client through the device application layer and, based on the communication result, causing the device driver to decode the bitstream data according to the decoding service issued by the client may include:

[0015] The device application layer obtains the decoding service issued by the client and notifies the device driver to perform decoding according to the decoding service.

[0016] The device driver decodes the bitstream data to obtain video frame data.

[0017] The process of communicating with the client through the device application layer and obtaining the algorithm analysis command sent by the client to instruct the algorithm application (APP) to perform algorithm analysis based on the communication result may include:

[0018] After the algorithm analysis task is successfully created, the device application layer returns a task handle to the client; the task handle is used to distinguish different analysis tasks and to indicate that the decoding channel has been created.

[0019] The device application layer receives algorithm analysis instructions returned by the client based on the task handle; wherein, the algorithm analysis instructions include an instruction handle and algorithm configuration parameters provided by the algorithm vendor; the instruction handle is used to indicate the identity of the bitstream data that needs to be analyzed by the algorithm, and the instruction handle may be one or more.

[0020] In an exemplary embodiment of this application, the step of performing algorithmic analysis on the acquired video frame data through the algorithm application APP may include:

[0021] The algorithm application APP directly calls one or more video frame data corresponding to the instruction handle from the device driver through the standard SDK interface;

[0022] The application uses the algorithm to perform algorithmic analysis on one or more video frame data based on the algorithm configuration parameters.

[0023] In an exemplary embodiment of this application, after performing algorithmic analysis on the acquired video frame data using the algorithm application APP, the method may further include:

[0024] The algorithm analysis results are assembled by the algorithm application APP according to the algorithm vendor's custom algorithm protocol, and the assembled algorithm analysis results are sent to the device application layer.

[0025] The device application layer reassembles the algorithm analysis results according to standard signaling and sends them to the client.

[0026] In an exemplary embodiment of this application, the method may further include: when the algorithm analysis results are reassembled through the device application layer, the algorithm analysis results are in a black-box state.

[0027] In an exemplary embodiment of this application, the device application layer and the algorithm application (APP) communicate via a message bus;

[0028] The device application layer and the device driver communicate via a message bus;

[0029] The device driver and the algorithm application (APP) communicate directly;

[0030] The message bus transmits messages in a message pass-through manner;

[0031] The algorithm package may also include any one or more of the following: algorithm model, algorithm library, and corresponding algorithm description file.

[0032] This application also provides a method for using an algorithm on a client-side basis, the method including:

[0033] The client connects to the edge computing node ECN and imports the algorithm package provided by the algorithm vendor according to the preset unified standard into the ECN by calling the standard SDK interface. The ECN receives the algorithm package through the device application layer, parses the algorithm package, obtains the algorithm application APP contained in the algorithm package, and performs algorithm analysis on the acquired video frame data through the algorithm application APP.

[0034] In an exemplary embodiment of this application, before performing algorithmic analysis on the acquired video frame data using the algorithm application APP, the method may further include:

[0035] The client obtains the Uniform Resource Locator (URL) address of the video stream of the video shooting device entered in the input window of the client, so that the device application layer can start the stream according to the video stream URL address and send the stream data obtained after the stream is started to the device driver.

[0036] The client communicates with the device application layer and sends decoding services to the device application layer, so that the device application layer notifies the ECN's device driver to decode the bitstream data according to the decoding services; and sends algorithm analysis instructions to the device application layer to instruct the algorithm application (APP) to perform algorithm analysis.

[0037] In an exemplary embodiment of this application, communicating with the device application layer through the client and sending an algorithm analysis instruction to the device application layer to instruct the algorithm application (APP) to perform algorithm analysis may include:

[0038] After the device driver completes decoding, the client receives a task handle returned by the device application layer; the task handle contains information indicating that decoding is complete.

[0039] The client returns an algorithm analysis instruction to the device application layer based on the task handle; wherein, the algorithm analysis instruction includes an instruction handle and algorithm configuration parameters provided by the algorithm vendor; the instruction handle is used to indicate the identity of the bitstream data that needs to be analyzed by the algorithm, and there are one or more instruction handles.

[0040] In an exemplary embodiment of this application, after performing algorithmic analysis on the acquired video frame data using the algorithm application APP, the method may further include:

[0041] The client receives the algorithm analysis results after the device application layer has reassembled them according to standard signaling. Before the second reassembly, the algorithm analysis results are first reassembled by the algorithm application (APP) according to the algorithm vendor's custom algorithm protocol and then sent to the device application layer.

[0042] This application also provides an edge computing node (ECN), which may include: a first processor and a first computer-readable storage medium; the first processor is capable of implementing the functions of the device application layer and the device driver; the first computer-readable storage medium stores a first instruction, which, when executed by the first processor, implements the ECN-side algorithm usage method described in any of the above.

[0043] This application also provides a client, which may include: a second processor and a second computer-readable storage medium; the second computer-readable storage medium stores a second instruction, which, when executed by the second processor, implements the client-side algorithm usage method described in any of the above.

[0044] This application also provides an algorithm usage system, which may include a client and an edge computing node (ECN); the ECN includes: a first processor and a first computer-readable storage medium; the first processor is capable of implementing device application layer and device driver functions; the client includes: a second processor and a second computer-readable storage medium; the first computer-readable storage medium stores a first instruction, and the second computer-readable storage medium stores a second instruction; after the client and the ECN are connected, when the first instruction is executed by the first processor and the second instruction is executed by the second processor, the algorithm usage method described above is implemented.

[0045] Compared with related technologies, the embodiments of this application are applied to the edge computing node ECN side, where the ECN may include a device application layer; the method may include: receiving an algorithm package through the device application layer, parsing the algorithm package, and obtaining the algorithm application APP contained in the algorithm package; the algorithm package is provided by an algorithm vendor according to a preset unified standard, and is connected to the ECN by a client, and imported into the ECN by the client calling the standard software development kit (SDK) interface; the algorithm application APP performs algorithm analysis on the acquired video frame data. This embodiment allows algorithm vendors to directly import algorithm packages without modifying existing algorithm interfaces, and without secondary development when interfacing with different algorithms.

[0046] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the application. Other advantages of this application can be realized and obtained by means of the solutions described in the description and the accompanying drawings. Attached Figure Description

[0047] The accompanying drawings are used to provide an understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0048] Figure 1 This is a flowchart illustrating the ECN-side algorithm usage method according to an embodiment of this application.

[0049] Figure 2 This is a schematic diagram of the frame structure of an ECN device according to an embodiment of this application;

[0050] Figure 3 This is a schematic diagram of the ECN software framework according to an embodiment of this application;

[0051] Figure 4 This is a schematic diagram of the algorithm analysis startup method according to an embodiment of this application;

[0052] Figure 5 This is a schematic diagram of the algorithm analysis result return method according to an embodiment of this application;

[0053] Figure 6 This is a flowchart illustrating the client-side algorithm usage method according to an embodiment of this application.

[0054] Figure 7 This is a block diagram of the ECN components in an embodiment of this application;

[0055] Figure 8 This is a block diagram of the client components in an embodiment of this application;

[0056] Figure 9The system composition block diagram used for the algorithm in the embodiments of this application;

[0057] Figure 10 This is a flowchart illustrating the algorithm usage method of the system used in the embodiments of this application. Detailed Implementation

[0058] This application describes several embodiments, but these descriptions are exemplary and not restrictive, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0059] This application includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this application may be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0060] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that it does not depend on such a specific order. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims concerning the method and / or process should not be limited to the steps performed in the written order, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments of this application.

[0061] This application provides an algorithm usage method applied to the ECN side of an edge computing node, wherein the ECN may include: a device application layer; such as Figure 1 As shown, the method may include steps S101-S102:

[0062] S101. Receive the algorithm package through the device application layer, parse the algorithm package, and obtain the algorithm application APP contained in the algorithm package; the algorithm package is provided by the algorithm vendor according to a preset unified standard, and is connected to the ECN on the client, and imported into the ECN by calling the standard software development kit (SDK) interface on the client.

[0063] S102. The obtained video frame data is analyzed by the algorithm application APP.

[0064] In an exemplary embodiment of this application, a standard operating environment is provided for intelligent analysis algorithms to run and be quickly replaced, eliminating the need for secondary development work for video shooting equipment (e.g., monitoring equipment, live streaming equipment, etc.) to interface with different algorithms.

[0065] In an exemplary embodiment of this application, ECN (Edge Computing Node) is an intelligent analysis embedded device that can quickly access video shooting equipment.

[0066] In an exemplary embodiment of this application, the ECN device is a complete embedded device that can be installed in any existing video shooting device and can be used by accessing the video stream; wherein, providing the ECN device with the accessible video stream URL (Uniform Resource Locator) address of the video shooting device enables the ECN device to access the video stream.

[0067] In exemplary embodiments of this application, as Figure 2 The diagram illustrates a framework of an ECN device. The hardware of an ECN device may include a processor and peripheral circuitry (e.g., a HiSilicon processor and peripheral circuitry). This processor and peripheral circuitry can be a System-on-Chip (SoC), and may also include, but are not limited to, memory, an eMMC (embedded multimedia controller), a USB (Universal Serial Bus) interface, an HDMI (High Definition Multimedia Interface) interface, a SATA (Serial Advanced Technology Accessory) interface, a hard drive interface, and a network port, among other interfaces. In an exemplary embodiment of this application, the ECN device can be connected to a video recording device, such as a surveillance system, via a network. The input to the ECN is a video stream; as long as a video stream address is provided, the ECN device can be used, making it convenient and widely applicable.

[0068] In an exemplary embodiment of this application, the ECN device can be compatible with algorithm packages provided by different algorithm vendors. In order to be compatible with different algorithm packages, the ECN device can provide standard interfaces for connecting to algorithm application apps. Only algorithm application apps that pass through these interfaces can be enabled in the ECN device.

[0069] In exemplary embodiments of this application, the algorithm package may also include, but is not limited to, any one or more of the following: algorithm model, algorithm library, and corresponding algorithm description file.

[0070] In an exemplary embodiment of this application, the proposed solution provides an ECN device that, when used in conjunction with a client, can resolve the difficulties faced by manufacturers of monitoring equipment in replacing intelligent analysis algorithms from different algorithm vendors, and the issue that algorithm vendors may need to provide SDK interfaces according to a unified standard. This enables rapid replacement of algorithms (e.g., facial recognition algorithms, temperature detection algorithms, and violation detection algorithms in monitoring equipment). Algorithm vendors do not need to modify existing algorithm interfaces; they only need to provide algorithm libraries and models according to standards, and then develop algorithm programs (such as the aforementioned algorithm application APP) themselves or have the algorithm vendors develop them to connect to the ECN device. ECN device manufacturers can directly import algorithm program packages when connecting to different algorithms without secondary development.

[0071] In exemplary embodiments of this application, the client may include, but is not limited to: demo (sample code / complete project code) programs, web (web page) clients, third-party device platforms, and PC (personal computer) clients (e.g., mobile phones, smart wearable devices, smart robots, tablets, etc.).

[0072] In an exemplary embodiment of this application, before the algorithm package is imported into the ECN via the client calling the standard software development kit (SDK) interface, the method may further include:

[0073] ECN equipment manufacturers provide algorithm vendors with a standard SDK interface and a standard method for packaging the algorithm packages, so that the algorithm vendors can package their respective algorithms according to the standard method, obtain the corresponding algorithm packages, and provide the algorithm packages through the standard SDK interface.

[0074] In an exemplary embodiment of this application, a new algorithm can be accessed by an ECN device in the following ways: the algorithm vendor provides an algorithm package according to a unified standard, which is then imported into the ECN device and parsed and stored by the device application layer.

[0075] In an exemplary embodiment of this application, an ECN device manufacturer can provide a standard SDK interface and a method for packaging intelligent analysis algorithms (based on the standard method described above). The algorithm manufacturer can package the algorithm application (APP), algorithm model, algorithm library, and corresponding description files according to this standard method to form the aforementioned algorithm package. The algorithm package is then imported into the ECN device via the standard SDK interface, allowing the ECN device to parse the algorithm package and record its relevant information. By enabling the corresponding algorithm application (APP) through the standard SDK interface, the corresponding algorithm can be used on the ECN device.

[0076] In an exemplary embodiment of this application, the software framework of the ECN can be as follows: Figure 3 As shown.

[0077] In an exemplary embodiment of this application, the device application layer can be a pure software program that can provide algorithm package management services (e.g., including but not limited to algorithm import, parsing, starting, stopping, etc.) and algorithm analysis task management (analysis task starting, stopping, etc.).

[0078] In an exemplary embodiment of this application, the ECN may further include a device driver. A device driver is specialized software that drives hardware. The role of the device driver in the ECN may be to provide necessary decoding and encoding interfaces (for example, video streams need to be decoded after entering the ECN, and image information in the reported algorithm analysis results needs to be encoded; these can be encapsulated by the ECN device manufacturer into standard SDK interfaces for algorithm application apps to call). The device driver layer ensures the normal operation of the hardware on the ECN device (including but not limited to the core processor, hard drive, USB, network card, etc.). The core processor provides the encoding and decoding service interfaces required by the ECN, and the device driver layer can provide basic service interfaces for use by the device application layer and the algorithm application app.

[0079] In an exemplary embodiment of this application, the device application layer and the algorithm application (APP) communicate via a message bus;

[0080] The device application layer and the device driver communicate via a message bus;

[0081] The device driver and the algorithm application (APP) communicate directly;

[0082] The message bus transmits messages in a message-transparent manner.

[0083] In an exemplary embodiment of this application, ECN ensures that communication between the device application layer and the device driver, as well as communication between the device application layer and the algorithm application (APP), uses a unified message bus. The message bus specifies a standard signaling interface that can start and stop algorithm analysis, and also provides a message pass-through method for transmitting custom messages from the algorithm or the device application layer, such as algorithm results.

[0084] In an exemplary embodiment of this application, the device application layer and the device driver interact via a message bus. When ECN is working, the device application layer obtains decoding resources from the device driver via the message bus and performs video decoding. The algorithm application (APP) obtains video frame data from the device driver (without needing to use the message bus) through the standard SDK interface provided by the device manufacturer and performs analysis. After obtaining the analysis results, the algorithm application (APP) needs to report the results to the device application layer through the encapsulated standard SDK interface (which needs to use the message bus).

[0085] In an exemplary embodiment of this application, after the video stream from any video shooting device is connected to the ECN, the ECN can use the existing algorithm application APP inside the device to analyze the connected video stream.

[0086] In an exemplary embodiment of this application, the following details how the video is decoded after the algorithm package is imported into the device application layer, how the algorithm analysis is initiated on the decoded video, and how the analysis results are returned.

[0087] In an exemplary embodiment of this application, before performing algorithmic analysis on the acquired video frame data using the algorithm application APP, the method further includes:

[0088] The device application layer initiates streaming based on the video stream's Uniform Resource Locator (URL) address and sends the obtained stream data to the device driver; the video stream URL address is input through the client's input window.

[0089] The device application layer communicates with the client, and based on the communication result, the device driver decodes the bitstream data according to the decoding service issued by the client, and obtains the algorithm analysis instruction sent by the client to instruct the algorithm application APP to perform algorithm analysis.

[0090] In an exemplary embodiment of this application, the user can input a video stream URL address through the input window of the client, and the client obtains the video stream URL address of the video shooting device input in the input window; the device application layer starts the streaming according to the video stream URL address, and sends the stream data obtained after starting the streaming to the device driver.

[0091] In an exemplary embodiment of this application, the step of communicating with the client through the device application layer and, based on the communication result, causing the device driver to decode the bitstream data according to the decoding service issued by the client may include:

[0092] The device application layer obtains the decoding service issued by the client and notifies the device driver to perform decoding according to the decoding service.

[0093] The device driver decodes the bitstream data to obtain video frame data.

[0094] In an exemplary embodiment of this application, the video stream URL address (video Uniform Resource Locator URL) is provided to the device application layer. The device application layer starts the stream through the video URL, decodes the stream data through the device driver, and generates video frame data after decoding, so that the algorithm application APP obtains the video frame data through the standard SDK interface.

[0095] In an exemplary embodiment of this application, the video URL is a simple address. If video stream data needs to be obtained from this address, streaming needs to be started, which can be done through the standard RTSP (Real-Time Streaming Protocol).

[0096] In exemplary embodiments of this application, as Figure 4 As shown, the device application layer communicates with the client, and obtains the algorithm analysis command sent by the client to instruct the algorithm application (APP) to perform algorithm analysis based on the communication result. This command may include:

[0097] After the algorithm analysis task is successfully created, the device application layer returns a task handle to the client; the task handle is used to distinguish different analysis tasks and to indicate that the decoding channel has been created.

[0098] The device application layer receives algorithm analysis instructions returned by the client based on the task handle; wherein, the algorithm analysis instructions include: an instruction handle and algorithm configuration parameters provided by the algorithm vendor; the instruction handle is used to indicate the identity of the bitstream data that needs to be analyzed by the algorithm, and the instruction handle may be one or more.

[0099] In an exemplary embodiment of this application, the task handle can be any custom handle.

[0100] In an exemplary embodiment of this application, the process of the client initiating algorithm analysis may include: the client adding a decoding service to the ECN, obtaining the task handle returned by the device application layer, sending the algorithm configuration parameters and instruction handle to the ECN device application layer through standard signaling, and the ECN device application layer notifying the algorithm application (APP) to start and begin executing the algorithm analysis task.

[0101] In an exemplary embodiment of this application, the aforementioned instruction handle can be an integer, which can be used to distinguish which specific video stream it is. One video URL can correspond to one instruction handle, and ECN can support the simultaneous creation of multiple algorithm analysis tasks.

[0102] In the exemplary embodiments of this application, the algorithm configuration parameters may be provided by the algorithm vendor, and these parameters may include different content depending on the algorithm, without detailed limitations. For example, for a face recognition algorithm, the algorithm configuration parameters may include: region coordinates, sensitivity, interpupillary distance, minimum face size, and maximum face size, etc.

[0103] In an exemplary embodiment of this application, the step of performing algorithmic analysis on the acquired video frame data through the algorithm application APP may include:

[0104] The algorithm application APP directly calls one or more video frame data corresponding to the instruction handle from the device driver through the standard SDK interface;

[0105] The application uses the algorithm to perform algorithmic analysis on one or more video frame data based on the algorithm configuration parameters.

[0106] In an exemplary embodiment of this application, since ECN can support the simultaneous establishment of multiple algorithm analysis tasks, it is possible to obtain multiple sets of video frame data (each command handle corresponds to a set of video frame data) according to the provided multiple command handles, and perform algorithm analysis on the multiple video frame data simultaneously.

[0107] In the exemplary embodiments of this application, the content analyzed by the algorithm varies depending on the algorithm. For example, for a face recognition algorithm, the algorithm analysis can identify different faces. For some behavior recognition algorithms, the algorithm analysis can include the identification of some behaviors, such as smoking behavior, area intrusion, checking for not wearing a mask, motion detection, etc.

[0108] In exemplary embodiments of this application, as Figure 5 As shown, after performing algorithmic analysis on the acquired video frame data using the aforementioned algorithm application APP, the method may further include:

[0109] The algorithm analysis results are assembled by the algorithm application APP according to the algorithm vendor's custom algorithm protocol, and the assembled algorithm analysis results are sent to the device application layer.

[0110] The device application layer reassembles the algorithm analysis results according to standard signaling and sends them to the client.

[0111] In an exemplary embodiment of this application, the process of sending algorithm results by the algorithm application (APP) may include: after the algorithm program assembles the algorithm results according to the algorithm protocol defined by the algorithm vendor, it sends them to the device application layer through the message bus, and the device application layer reassembles the algorithm results according to standard signaling and sends them to the designated client.

[0112] In an exemplary embodiment of this application, there can be multiple clients connected to the ECN device, which can simultaneously update and apply algorithms provided by multiple algorithm vendors. Therefore, when returning analysis results, different analysis results are returned to the corresponding clients.

[0113] In an exemplary embodiment of this application, the method may further include: when the algorithm analysis results are reassembled through the device application layer, the algorithm analysis results are in a black-box state.

[0114] In an exemplary embodiment of this application, the principle of black-boxing of ECN messages includes: the ECN device message bus provides a message pass-through method, so that the ECN device application layer does not need to parse messages during the transmission of messages between the client and the algorithm application (APP). The assembly and parsing of messages containing algorithm configuration parameters and messages containing algorithm analysis results can be performed by the client and the algorithm application (APP), so the ECN does not need to parse the above parameters during this process.

[0115] In an exemplary embodiment of this application, the algorithm configuration parameter message is treated as a black box: it supports algorithm vendors sending custom algorithm configuration parameters from the client to the device application layer of the ECN via standard signaling. The device application layer does not perform any internal processing on the algorithm configuration parameter message, but instead passes it through the message bus to the algorithm application (APP), which then parses the message.

[0116] In an exemplary embodiment of this application, the algorithm analysis result message is black-boxed: after the algorithm application APP parses the bitstream to obtain the algorithm analysis result, it sends the result to the device application layer through the message bus. The device application layer does not process the algorithm analysis result and sends it to the designated client.

[0117] The exemplary embodiments of this application include at least the following advantages:

[0118] 1. Algorithm vendors do not need to modify existing algorithm interfaces; they only need to provide algorithm libraries and models according to the standards. Then, users can develop their own algorithm application apps, either independently or through the algorithm vendor, to connect to ECN devices.

[0119] 2. ECN device manufacturers can directly import algorithm programs when integrating with different algorithms, without the need for secondary development.

[0120] This application also provides an algorithm usage method applied to the client side, such as... Figure 6 As shown, the method may include step S201:

[0121] S201. Connect the client to the edge computing node ECN, and import the algorithm package provided by the algorithm vendor according to the preset unified standard into the ECN by calling the standard SDK interface through the client. The ECN's device application layer receives the algorithm package, parses the algorithm package, obtains the algorithm application APP contained in the algorithm package, and performs algorithm analysis on the obtained video frame data through the algorithm application APP.

[0122] In an exemplary embodiment of this application, before performing algorithmic analysis on the acquired video frame data using the algorithm application APP, the method may further include:

[0123] The client obtains the Uniform Resource Locator (URL) address of the video stream of the video shooting device entered in the input window of the client, so that the device application layer can start the stream according to the video stream URL address and send the stream data obtained after the stream is started to the device driver.

[0124] The client communicates with the device application layer and sends decoding services to the device application layer, so that the device application layer notifies the ECN's device driver to decode the bitstream data according to the decoding services; and sends algorithm analysis instructions to the device application layer to instruct the algorithm application (APP) to perform algorithm analysis.

[0125] In an exemplary embodiment of this application, communicating with the device application layer through the client and sending an algorithm analysis instruction to the device application layer to instruct the algorithm application (APP) to perform algorithm analysis may include:

[0126] After the device driver completes decoding, the client receives a task handle returned by the device application layer; the task handle contains information indicating that decoding is complete.

[0127] The client returns an algorithm analysis instruction to the device application layer based on the task handle; wherein, the algorithm analysis instruction includes an instruction handle and algorithm configuration parameters provided by the algorithm vendor; the instruction handle is used to indicate the identity of the bitstream data that needs to be analyzed by the algorithm, and there are one or more instruction handles.

[0128] In an exemplary embodiment of this application, after performing algorithmic analysis on the acquired video frame data using the algorithm application APP, the method may further include:

[0129] The client receives the algorithm analysis results after the device application layer has reassembled them according to standard signaling. Before the second reassembly, the algorithm analysis results are first reassembled by the algorithm application (APP) according to the algorithm vendor's custom algorithm protocol and then sent to the device application layer.

[0130] In the exemplary embodiments of this application, any algorithm usage method embodiment measured by the ECN device is applicable to the algorithm usage embodiment on the client side, and will not be described in detail here.

[0131] This application embodiment also provides an edge computing node ECN 1, such as Figure 7 As shown, it may include: a first processor 11 and a first computer-readable storage medium 12; the first processor 11 is capable of implementing the functions of the device application layer and the device driver; the first computer-readable storage medium 12 stores a first instruction, which, when executed by the first processor 11, implements the algorithm usage method described in any of the above.

[0132] In the exemplary embodiments of this application, any algorithm used in the ECN device measurement method embodiment is applicable to the ECN device embodiment, and will not be described in detail here.

[0133] This application embodiment also provides a client 2, such as Figure 8 As shown, it may include: a second processor 21 and a second computer-readable storage medium 22; the second computer-readable storage medium 22 stores a second instruction, which, when executed by the second processor 21, implements the algorithm usage method of the ECN device measurement described in any of the above.

[0134] In the exemplary embodiments of this application, any algorithm used in the ECN device measurement method embodiment is applicable to the client embodiment, and will not be described in detail here.

[0135] This application also provides an algorithm usage system 3, such as... Figure 9As shown, it may include a client 2 and an edge computing node ECN 1; the ECN includes: a first processor and a first computer-readable storage medium; the first processor is capable of implementing the functions of the device application layer and device driver; the client includes: a second processor and a second computer-readable storage medium; the first computer-readable storage medium stores a first instruction, and the second computer-readable storage medium stores a second instruction. After the client and the ECN are connected, when the first instruction is executed by the first processor and the second instruction is executed by the second processor, the client-side algorithm usage method described above is implemented.

[0136] In exemplary embodiments of this application, as Figure 10 As shown, the execution steps of this system may include S301-S303:

[0137] S301. Connect the client to the ECN, and import the algorithm package provided by the algorithm vendor according to the preset unified standard into the ECN by calling the standard SDK interface through the client;

[0138] S302. Receive the algorithm package through the device application layer of the ECN, parse the algorithm package, and obtain the algorithm application APP contained in the algorithm package;

[0139] S303. The obtained video frame data is analyzed by the algorithm application APP.

[0140] In the exemplary embodiments of this application, any algorithm used in the ECN device measurement method embodiment is applicable to the system embodiment, and will not be described in detail here.

[0141] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

Claims

1. A method for using an algorithm, characterized in that, Applied to the ECN side of an edge computing node, wherein the ECN includes: a device application layer; the method includes: The device application layer receives the algorithm package and parses it to obtain the algorithm application APP contained in the algorithm package. The algorithm package is provided by the algorithm vendor according to a preset unified standard and is imported into the ECN by the client calling the standard software development kit (SDK) interface when the client connects to the ECN. The algorithm is applied to the APP to perform algorithmic analysis on the acquired video frame data; The ECN provides a message pass-through method through the message bus. The device application layer and the algorithm application APP communicate through the message bus, so that the device application layer does not need to parse messages when transmitting messages between the client and the algorithm application APP.

2. The method of using the algorithm according to claim 1, characterized in that, Before the algorithm package is imported into the ECN via the client by calling the standard software development kit (SDK) interface, the method further includes: ECN equipment manufacturers provide algorithm vendors with a standard SDK interface and a standard method for packaging the algorithm packages, so that the algorithm vendors can package their respective algorithms according to the standard method, obtain the corresponding algorithm packages, and provide the algorithm packages through the standard SDK interface.

3. The method of using the algorithm according to claim 1, characterized in that, The ECN further includes: a device driver; before performing algorithmic analysis on the acquired video frame data through the algorithm application APP, the method further includes: The device application layer initiates streaming based on the video stream's Uniform Resource Locator (URL) address and sends the obtained stream data to the device driver; the video stream URL address is input through the client's input window. The device application layer communicates with the client, and based on the communication result, the device driver decodes the bitstream data according to the decoding service issued by the client, and obtains the algorithm analysis instruction sent by the client to instruct the algorithm application APP to perform algorithm analysis.

4. The method of using the algorithm according to claim 3, characterized in that, The step of communicating with the client through the device application layer, and instructing the device driver to decode the bitstream data according to the decoding service issued by the client based on the communication result, includes: The device application layer obtains the decoding service issued by the client and notifies the device driver to perform decoding according to the decoding service. The device driver decodes the bitstream data to obtain video frame data; The process involves communicating with the client through the device application layer, and obtaining, based on the communication results, algorithm analysis instructions sent by the client to instruct the algorithm application (APP) to perform algorithm analysis, including: After the algorithm analysis task is successfully created, the device application layer returns a task handle to the client; the task handle is used to distinguish different analysis tasks and to indicate that the decoding channel has been created. The device application layer receives algorithm analysis instructions returned by the client based on the task handle; wherein, the algorithm analysis instructions include an instruction handle and algorithm configuration parameters provided by the algorithm vendor; the instruction handle is used to indicate the identity of the bitstream data that needs to be analyzed by the algorithm, and the instruction handle may be one or more.

5. The method of using the algorithm according to claim 4, characterized in that, The algorithmic analysis of the acquired video frame data by the APP using the algorithm includes: The algorithm application APP directly calls one or more video frame data corresponding to the instruction handle from the device driver through the standard SDK interface; The application uses the algorithm to perform algorithmic analysis on one or more video frame data based on the algorithm configuration parameters.

6. The method of using the algorithm according to any one of claims 1-5, characterized in that, After performing algorithmic analysis on the acquired video frame data using the aforementioned algorithm application APP, the method further includes: The algorithm analysis results are assembled by the algorithm application APP according to the algorithm vendor's custom algorithm protocol, and the assembled algorithm analysis results are sent to the device application layer. The algorithm analysis results are reassembled by the device application layer according to standard signaling and then sent to the client. When the algorithm analysis results are reassembled by the device application layer, the algorithm analysis results are in a black box state.

7. A method for using an algorithm, characterized in that, Applied to the client side, the method includes: The client connects to the edge computing node ECN and imports the algorithm package provided by the algorithm vendor according to the preset unified standard into the ECN by calling the standard SDK interface. The ECN receives the algorithm package through the device application layer, parses the algorithm package, obtains the algorithm application APP contained in the algorithm package, and performs algorithm analysis on the obtained video frame data through the algorithm application APP. The ECN provides a message pass-through method through the message bus. The device application layer and the algorithm application APP communicate through the message bus, so that the device application layer does not need to parse messages when transmitting messages between the client and the algorithm application APP.

8. The method of using the algorithm according to claim 7, characterized in that, Before performing algorithmic analysis on the acquired video frame data using the aforementioned algorithm application APP, the method further includes: The client obtains the video stream Uniform Resource Locator (URL) address of the video shooting device entered in the input window of the client, so that the device application layer can start the stream according to the video stream URL address and send the stream data obtained after the stream is started to the device driver. The client communicates with the device application layer and sends decoding services to the device application layer, so that the device application layer notifies the ECN's device driver to decode the bitstream data according to the decoding services; and sends algorithm analysis instructions to the device application layer to instruct the algorithm application (APP) to perform algorithm analysis.

9. The method of using the algorithm according to claim 8, characterized in that, The client communicates with the device application layer, sending algorithm analysis instructions to the device application layer to instruct the algorithm application (APP) to perform algorithm analysis, including: After the device driver completes decoding, the client receives a task handle returned by the device application layer; the task handle contains information indicating that decoding is complete. The client returns an algorithm analysis instruction to the device application layer based on the task handle; wherein, the algorithm analysis instruction includes an instruction handle and algorithm configuration parameters provided by the algorithm vendor; the instruction handle is used to indicate the identity of the bitstream data that needs to be analyzed by the algorithm, and there are one or more instruction handles.

10. An edge computing node (ECN), characterized in that, include: A first processor and a first computer-readable storage medium; the first processor is capable of implementing the functions of the device application layer and device driver; A first computer-readable storage medium stores a first instruction, which, when executed by the first processor, implements the algorithm usage method as described in any one of claims 1-6.

11. A client, characterized in that, include: Second processor and second computer-readable storage medium; The second computer-readable storage medium stores a second instruction that, when executed by the second processor, implements the algorithm usage method as described in any one of claims 7-9.

12. An algorithm-using system, characterized in that, The system includes a client and an edge computing node (ECN). The ECN includes a first processor and a first computer-readable storage medium. The first processor is capable of implementing device application layer and device driver functions. The client includes a second processor and a second computer-readable storage medium. The first computer-readable storage medium stores a first instruction, and the second computer-readable storage medium stores a second instruction. After the client and the ECN are connected, when the first instruction is executed by the first processor and the second instruction is executed by the second processor, the algorithm usage method described in any one of claims 1-6 and any one of claims 7-9 is implemented.

Images