A service separation platform based on video service and a service separation device

Abstract

The application provides a service separation platform and device based on a video service. The application comprises an application layer configured to send a task instruction to an intelligent service layer; the intelligent service layer is configured to send a data request to a video service layer according to the task instruction; the video service layer is configured to send to-be-processed data to the intelligent service layer based on the data request; the intelligent service layer is configured to process the to-be-processed data according to an intelligent algorithm specified by the task instruction, generate an algorithm result, and send the algorithm result to the video service layer; and the video service layer is configured to complete a screenshot operation on the algorithm result according to a screenshot instruction of the application layer, and return a screenshot result of the algorithm result to the application layer. In the above manner, the service separation platform decouples common functions such as video data management and screenshot from services, has a more distinct hierarchy, has less code maintenance, is easier to extend functions, builds a new platform more quickly and efficiently, has a higher code reuse rate between platforms, and is conducive to integrating service resources of a product line.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a service separation platform and service separation device based on video services. Background Technology

[0002] With the continuous development of deep learning technology, more and more artificial intelligence algorithms are being deployed in mobile and embedded devices, making significant contributions to the digital and intelligent upgrading of various industries. As smart network cameras become more widespread, the tasks of video data analysis and processing are increasingly moving forward, placing higher demands on the front-end processing capabilities of network cameras. In many scenarios, video data is highly complex, posing a significant challenge to real-time video data analysis and processing capabilities. Therefore, how to achieve a more efficient and secure video service module in intelligent business has become a common problem faced by the industry.

[0003] A complete business solution typically requires the support of multiple intelligent algorithms. The inputs and outputs of these algorithms are interdependent, and the final results provide the data foundation for upper-layer application development. In current business solutions, video data and the scheduling of intelligent algorithms are coupled together. Each intelligent algorithm manages the video data independently, which can easily lead to errors such as one algorithm releasing data while another is still using it, resulting in data access errors. Secondly, if each intelligent algorithm copies video data while using it, it will consume a significant amount of device memory, wasting memory and reducing memory utilization efficiency. Summary of the Invention

[0004] This application provides a service separation platform and service separation device based on video services.

[0005] This application provides a service separation platform based on video services, the service separation platform comprising:

[0006] The application layer is used to send task instructions to the intelligent business layer;

[0007] The intelligent service layer is used to send data requests to the video service layer according to task instructions;

[0008] The video service layer is used to send data to be processed to the intelligent business layer based on the data request;

[0009] The intelligent service layer is used to process the data to be processed according to the intelligent algorithm specified by the task instruction, generate the algorithm result, and send it to the video service layer.

[0010] The video service layer is used to perform a capture operation on the algorithm result according to the capture command of the application layer, and return the capture result of the algorithm result to the application layer.

[0011] The application layer is used to determine whether the image capture conditions are met based on the algorithm results returned by the intelligent business layer. If so, it sends an image capture command to the video service layer.

[0012] The video service layer is configured to encode the image corresponding to the capture frame number in the capture instruction, overlay the algorithm result onto the image, and return the overlay algorithm result image to the application layer.

[0013] The video service layer includes a service and interaction layer, as well as a data and image capture management layer.

[0014] The service and interaction layer is used to obtain the screenshot instruction from the application layer and send the screenshot instruction to the data and screenshot management layer.

[0015] The data and image capture management layer is used to encode a first image from the video stream according to the image capture frame sequence number in the image capture instruction, use the algorithm result to perform image cutout on the video stream to obtain the cutout video stream, and encode a second image; superimpose the second image onto the first image, and return the superimposed image to the application layer by calling the callback function of the application layer through the service and interaction layer.

[0016] The video service layer also includes a platform encapsulation layer;

[0017] The platform encapsulation layer is used to provide a unified transmission interface for the intelligent service layer.

[0018] The video service layer also includes a configuration management layer;

[0019] The configuration management layer is used to provide configuration information to the platform encapsulation layer, the service and interaction layer, and the data and screenshot management layer.

[0020] The configuration information includes the acquisition resolution, encoding format, encoding resolution and / or motion detection resolution configuration information of the platform encapsulation layer; the interface enable configuration and capability set configuration configuration information of the service and interaction layer; and the encoding scheme, the number of video stream data transferred in the queue, the number of images cached and / or the number of video stream data cached in the queue configuration information of the data and image capture management layer.

[0021] The service and interaction layer is used to provide public video functions of the video service layer. The public video functions include one or more video functions such as occlusion detection, defocus detection, encoding, bitstream packaging, and analog output.

[0022] The business separation platform also includes a presentation layer;

[0023] The presentation layer is used to receive the image capture result of the algorithm result from the application layer, wherein the application layer adjusts the image parameters and / or image format of the image capture result of the algorithm result.

[0024] The presentation layer is used to send working mode setting information to the application layer;

[0025] The application layer is used to send task instructions to the intelligent business layer according to the setting information of the working mode. The task instructions include the working mode of the image capture function, and the working mode includes real-time image capture or non-real-time image capture.

[0026] This application also provides a service separation device, which is equipped with the above-mentioned service separation platform and performs service processing on video / images through the service separation platform.

[0027] The beneficial effects of this application are as follows: The business separation platform includes: an application layer for sending task instructions to the intelligent business layer; the intelligent business layer for sending data requests to the video service layer according to the task instructions; the video service layer for sending data to be processed to the intelligent business layer based on the data requests; the intelligent business layer for processing the data to be processed according to the intelligent algorithm specified by the task instructions, generating algorithm results, and sending them to the video service layer; and the video service layer for performing a capture operation on the algorithm results according to the capture instructions from the application layer, and returning the capture results of the algorithm results to the application layer. Through this method, the business separation platform decouples common functions such as video data management and capture from business operations, resulting in clearer layers, less code maintenance, easier functional expansion, faster and more efficient construction of new platforms, higher code reusability between platforms, and facilitates the integration of business resources from the finished product line. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0029] Figure 1This is a schematic diagram of the framework of an embodiment of the video service-based service separation platform provided in this application;

[0030] Figure 2 This is a schematic diagram of the module relationships of the video service layer provided in this application;

[0031] Figure 3 This is a schematic diagram of a framework of an embodiment of the business separation device provided in this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0033] This application addresses the shortcomings of current intelligent solutions, such as poor code compatibility, poor scalability, and weak cross-platform support, caused by the over-coupling of common functions like video data management and image capture with intelligent business logic. It proposes a video service platform based on business separation. This platform decouples common video functions from intelligent business logic, breaking them down into modules such as a platform encapsulation layer, a service and interaction layer, a data and image capture management layer, and a configuration management layer. This results in a clearer hierarchy, less code maintenance, and easier functional expansion.

[0034] The video service is a comprehensive module that provides video data sending, video transmission, video encoding, video decoding, and video data destruction services for the intelligent business layer and application layer. It has advantages such as high cohesion, low coupling, high memory utilization, efficient data transmission, and ease of function expansion and maintenance.

[0035] The business separation platform of this application decouples common functions such as video data management and image capture from business operations, and decomposes them into modules such as platform encapsulation layer, service and interaction layer, data and image capture management layer, and configuration management layer. The layers are clearer, the amount of code maintenance is less, it is easier to expand functions, the new platform is built faster and more efficiently, the code reuse rate between platforms is higher, and it is conducive to integrating the business resources of the finished product line.

[0036] Please refer to the details. Figure 1 and Figure 2 , Figure 1 This is a schematic diagram of the framework of an embodiment of the video service-based service separation platform provided in this application. Figure 2 This is a schematic diagram of the module relationships of the video service layer provided in this application.

[0037] like Figure 1As shown, the business separation platform 100 of this application includes an application layer 11, a video service layer 12, an intelligent business layer 13, and a web presentation layer 14.

[0038] Specifically, the application layer 11 is used to send task instructions to the intelligent service layer 13; the intelligent service layer 13 is used to send data requests to the video service layer 12 according to the task instructions; the video service layer 12 is used to send data to be processed to the intelligent service layer 13 based on the data requests; the intelligent service layer 13 is used to process the data to be processed according to the intelligent algorithm specified by the task instructions, generate algorithm results, and send them to the video service layer 12; the video service layer 12 is used to complete the image capture operation on the algorithm results according to the image capture instructions of the application layer 11, and return the image capture results of the algorithm results to the application layer 11.

[0039] like Figure 2 As shown, the video service layer 12 includes a service and interaction layer 121, a data and image capture management layer 122, a platform encapsulation layer 123, and a configuration management layer 124.

[0040] The working principles of each module in the video service layer 12 are described below:

[0041] The main function of the platform encapsulation layer 123 is to encapsulate the differences between different platforms, provide a unified interface to the intelligent business layer 13, and ultimately call the platform SDK. The platform encapsulation layer 123 is the lowest layer of the entire program, responsible for communication between upper-layer applications and the hardware platform. It includes functions such as input, output, encoding, setting OSD overlay, setting ROI regions, acquiring YUV data, acquiring bitstreams, and system common capabilities.

[0042] The main function of the service and interaction layer 121 is to provide common functions of the video service layer 12 and to interact with the application layer 11. This includes cross-platform common video business logic, communication and shared memory management modules, platform communication node configuration, providing interfaces to the application layer, and internal and external service nodes. The cross-platform common business logic encapsulates the differences at the platform's underlying layers, abstracting common video functions, including occlusion detection, defocus detection, encoding, stream packaging, and analog output. The communication and shared memory management module includes intra-process communication configuration and inter-process communication configuration. Platform communication node configuration includes the number of kernels, processes, and threads. Providing interfaces to the application layer mainly includes creating interaction interfaces with applications, such as creating a video encoding device interface (creating encoding communication nodes with applications, shared memory, registering callback functions with applications, and creating the system); and creating a video detection interface (creating detection communication nodes with applications, registering callback functions with applications, and creating threads for managing application results). The internal and external service layers include an internal service layer and an external service layer. The internal service layer includes encoding and detection, while the external service layer includes autofocus.

[0043] The main functions of the data and image capture management layer 122 are data management and image capture operations. Data management includes the application and release of YUV data, receiving image capture data sent by the application layer 11, and releasing captured data. Image capture functions include image encoding, decoding, and overlaying OSD information. Acquiring and releasing YUV data are achieved by calling the unified interface exposed by the platform encapsulation layer. After the image capture operation is completed, the encoded image data is sent to the service and interaction layer. The service and interaction layer 121 pushes the image data to the application layer 11 by calling the callback function of the application layer 11. After processing by the application layer 11, the image data is sent to the web client for display.

[0044] The main function of the configuration management layer 124 is to provide configuration management for the service and interaction layer 121, the data and image capture management layer 122, and the platform encapsulation layer 123. For the platform encapsulation layer 123, this includes configurations for acquisition resolution, encoding format, encoding resolution, and motion detection resolution; for the service and interaction layer 121, this includes interface enable configuration and capability set configuration; and for the data management and image capture management layer 122, this includes configurations for encoding schemes, the number of large image YUV data streams in the queue, the number of JPEG caches, and the number of small image YUV data caches in the queue.

[0045] The following section uses the video data processing flow within the entire service separation platform 100 to further illustrate the working principles of each module in the service separation platform 100:

[0046] First, users or clients set the working mode of the image capture function and enable intelligent services in the web presentation layer 14. There are two main working modes: real-time image capture and non-real-time image capture. Real-time image capture means that the intelligent algorithm performs real-time image capture operations on targets that meet the capture conditions during the working process, regardless of whether the data is optimal. Non-real-time image capture means that the intelligent algorithm further analyzes and processes targets that meet the capture conditions during the working process, selecting the optimal data for image capture. Intelligent services are actual business scenarios with specific functions implemented using intelligent algorithms, such as intelligent vehicle detection, intelligent pedestrian flow detection, intelligent vehicle speed detection, and intelligent water flow detection.

[0047] Application layer 11 sets the working rules of intelligent business layer 13. The rules set are different depending on the intelligence enabled. For example, in vehicle detection intelligence, rule lines can be set, and exceeding the rule lines is considered as illegal driving. In pedestrian flow detection intelligence, detection areas can be set to detect pedestrian flow in specific areas. In perimeter detection intelligence, warning lines and warning areas can be set as the boundary lines of the perimeter.

[0048] After the intelligent business layer 13 starts working, it sends a message to the data and capture management layer 122 in the video service layer 12 to request data transmission. Upon receiving the request, the data and capture management layer 122 calls the data retrieval interface of the platform encapsulation layer 123 to send the YUV data to the intelligent business layer 13. After receiving the data, the intelligent business layer 13 calls the corresponding algorithm library to start intelligent processing of the data, obtains the corresponding intelligent analysis results, sends the intelligent analysis results to the video service layer 12, and simultaneously reports the event to the application layer 11.

[0049] Application layer 11 judges the events reported by intelligent business layer 13. If the conditions for capturing the image are met, it sends the image capture command and the information to be superimposed on the image, such as timestamp, black border information, etc., to service and interaction layer 121 of video service layer 12.

[0050] The video service layer 12 receives the algorithm results sent by the intelligent business layer 13 and temporarily caches the results in the result queue Que1.

[0051] After receiving the image capture command, the service and interaction layer 121 of the video service layer 12 notifies the data and image capture management layer 122 of the frame sequence number to be captured. The data and image capture management layer 122 then performs the encoding operation on the frame data of the corresponding frame sequence number. First, the corresponding data information is overlaid on the YUV data. Then, the YUV data is encoded into JPEG data, and the large JPEG data is stored in queue Que2. At the same time, using the target detection box information Que1 sent by the intelligent business layer 13, the YUV data is subjected to a cutout operation, and the small YUV data is encoded into JPEG data and stored in the corresponding data queue Que3.

[0052] After the data and image capture management layer 122 completes the image capture operation, it notifies the service and interaction layer 121. The service and interaction layer 121 then calls the callback function registered in the video service layer 12 by the application layer 11 to notify the application layer 11 of the encoding result.

[0053] After receiving the encoded image sent by the video service layer 12, the application layer 11 performs certain processing on the image, including adjusting the image resolution and compression format, and then sends it to the web presentation layer 14 via network protocol.

[0054] The web presentation layer 14 is responsible for displaying the data, including real-time target trajectory bounding boxes and captured image data. The real-time target trajectory bounding boxes come from the real-time results obtained by the intelligent business layer 13 after detecting the target, and are sent to the application layer 11 and then to the web presentation layer 14. The captured image data comes from the captured image data of the video service layer 12, is processed by the application layer 11, and then sent to the web presentation layer 14.

[0055] In this embodiment, the business separation platform 100 includes: an application layer 11 for sending task instructions to an intelligent business layer 13; the intelligent business layer 13 for sending data requests to a video service layer 12 according to the task instructions; the video service layer 12 for sending data to be processed to the intelligent business layer 13 based on the data requests; the intelligent business layer 13 for processing the data to be processed according to the intelligent algorithm specified by the task instructions, generating algorithm results, and sending them to the video service layer 12; and the video service layer 12 for performing a capture operation on the algorithm results according to the capture instructions from the application layer 11, and returning the capture results of the algorithm results to the application layer 11. Through this method, the business separation platform 100 decouples common functions such as video data management and capture from business operations, resulting in clearer layers, less code maintenance, easier functional expansion, faster and more efficient new platform construction, higher code reusability between platforms, and facilitates the integration of business resources from the finished product line.

[0056] This application proposes a video service method based on business separation, which decouples common functions such as video data management and image capture from intelligent algorithms. These functions are decomposed into modules such as a platform encapsulation layer, a service and interaction layer, a data and image capture management layer, and a configuration management layer, which are managed separately. This achieves decoupling of common video functions and intelligent business, resulting in higher code reusability across platforms, faster construction of new platform frameworks, and improved scalability and maintainability of the video service method. Furthermore, it proposes a configuration file management method decoupled from modules, allowing different functional modules to share the same set of configuration files for management, simplifying the configuration management process.

[0057] Based on the video service-based service separation platform described in the above embodiments, this application also proposes a service separation device, for details please refer to [link to details]. Figure 3 , Figure 3 This is a schematic diagram of a framework of an embodiment of the business separation device provided in this application.

[0058] like Figure 3 As shown, the service separation device 300 of this application embodiment is equipped with the service separation platform 31 shown in the above embodiment, and its specific framework will not be described again here. Specifically, the service separation device 300 performs service processing on video / images through the service separation platform 31.

[0059] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A service separation platform based on video services, characterized in that, The business separation platform includes: The application layer is used to send task instructions to the intelligent business layer; The intelligent service layer is used to send data requests to the video service layer according to task instructions; The video service layer is used to send data to be processed to the intelligent business layer based on the data request; The intelligent service layer is used to process the data to be processed according to the intelligent algorithm specified by the task instruction, generate the algorithm result, and send it to the video service layer. The video service layer is used to perform a capture operation on the algorithm result according to the capture command of the application layer, and return the capture result of the algorithm result to the application layer; The video service layer includes a service and interaction layer, as well as a data and image capture management layer; The service and interaction layer is used to provide public video functions of the video service layer and to interact with the application layer. The public video functions include one or more video functions such as occlusion detection, defocus detection, encoding, bitstream packaging, and analog output. The data and image capture management layer is used for data management and image capture operations. Specifically, the data and image capture management layer is used to encode a first image from the video stream according to the image capture frame number in the image capture instruction; to perform image matting on the video stream using the algorithm result to obtain a matted video stream; and to encode a second image; to overlay the second image onto the first image; and to return the overlaid image to the application layer by calling a callback function of the application layer through the service and interaction layer. The video service layer also includes a platform encapsulation layer; The platform encapsulation layer is used to provide a unified transmission interface for the intelligent service layer; The video service layer also includes a configuration management layer; The configuration management layer is used to provide configuration information for the service and interaction layer, the data and image capture management layer, and the platform encapsulation layer.

2. The business separation platform according to claim 1, characterized in that, The application layer is used to determine whether the image capture conditions are met based on the algorithm results returned by the intelligent service layer. If so, it sends an image capture command to the video service layer.

3. The business separation platform according to claim 2, characterized in that, The video service layer is used to encode the image corresponding to the capture frame sequence number in the capture instruction, overlay the algorithm result onto the image, and return the overlay algorithm result image to the application layer.

4. The business separation platform according to claim 3, characterized in that, The service and interaction layer is used to obtain the screenshot instruction from the application layer and send the screenshot instruction to the data and screenshot management layer.

5. The business separation platform according to claim 4, characterized in that, The configuration information includes the acquisition resolution, encoding format, encoding resolution and / or motion detection resolution configuration information of the platform encapsulation layer; the interface enable configuration and capability set configuration configuration information of the service and interaction layer; and the encoding scheme, the number of video stream data transferred in the queue, the number of images cached and / or the number of video stream data cached in the queue configuration information of the data and image capture management layer.

6. The business separation platform according to claim 3, characterized in that, The service and interaction layer is used to provide public video functions for the video service layer.

7. The business separation platform according to claim 1, characterized in that, The business separation platform also includes a presentation layer; The presentation layer is used to receive the image capture result of the algorithm result from the application layer, wherein the application layer adjusts the image parameters and / or image format of the image capture result of the algorithm result.

8. The business separation platform according to claim 7, characterized in that, The presentation layer is used to send working mode setting information to the application layer; The application layer is used to send task instructions to the intelligent business layer according to the setting information of the working mode. The task instructions include the working mode of the image capture function, and the working mode includes real-time image capture or non-real-time image capture.

9. A business separation device, characterized in that, The service separation device is equipped with the service separation platform according to any one of claims 1 to 8, and performs service processing on video / images through the service separation platform.

Images