Development system and processing method based on intelligent IPC efficient AI customization

Abstract

The invention relates to a development system and a processing method based on intelligent IPC efficient AI customization. An AI open framework is introduced; service standardization and communication standardization are carried out on the basis of the AI open framework; carrying out external packaging frame sending service on the basis of service standardization and communication standardization; and finally, outputting the document. According to the method, redesign is carried out on the basis of a traditional IPC, a basic process, a media processing process and an algorithm processing process are optimized, and an additional processing process is added. The basic process and the media processing process provide various basic services, such as an alarm service, a mail sending service, a call making service, an alert service, a snapshot service and a video streaming service, and support a third party to perform function development according to specific requirements; the algorithm processing process and the additional processing process provide standard development rules, allow a third party to replace the self-research process, and realize iteration and optimization of the algorithm.

Description

Technical Field

[0001] The present invention belongs to the field of edge algorithm technology, and in particular to a development system and processing method based on intelligent IPC efficient AI customization. Background Art

[0002] With the advancement and development of technology, many traditional industries are also moving towards intelligence, which is the general trend, including electricity, finance, education, operators, railways, etc.

[0003] These are areas with strong industry application specificity. If you start from hardware design and then develop intelligent algorithms and industry applications, and integrate different algorithms into the same system or application, you must do a lot of custom development and adaptation work, which will extend the development cycle, increase costs, and reduce the flexibility and maintainability of the system. Therefore, enabling traditional security equipment to quickly embed these algorithms has become the core of the competitiveness of industry equipment.

[0004] The existing "embedded method" has the following defects and deficiencies in the development of intelligent security equipment:

[0005] 1. Custom development and adaptation work is arduous: Existing methods usually require developers to carry out a lot of custom development and adaptation work according to the requirements of different algorithms when starting from hardware design. This process is not only time-consuming, but also requires a lot of resources, which significantly prolongs the development cycle.

[0006] 2. High cost: Since each algorithm needs to be integrated and debugged separately, the overall development cost increases. The adaptation work between hardware and algorithm and the integrated debugging bring additional economic burden.

[0007] 3. Low system flexibility: Existing methods are usually tailored for specific algorithms, resulting in the system lacking sufficient flexibility when facing new algorithms or updates. This limitation restricts the system's ability to expand and upgrade.

[0008] 4. Difficulty in maintenance: Due to the customization and complexity of the system, problems are prone to occur during maintenance and updates, which increases the complexity and difficulty of maintenance. This may also affect the stability and reliability of the system.

[0009] 5. Difficulty in integrating algorithm manufacturers and hardware manufacturers: In existing methods, the collaboration between algorithm manufacturers and hardware equipment manufacturers is often not close enough, resulting in an unsmooth algorithm integration process and difficulty in quickly launching intelligent equipment that meets industry needs. Summary of the invention

[0010] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a development system and processing method based on intelligent IPC efficient AI customization, which can maintain the stable operation of the power grid, quickly and accurately provide the optimal load transfer plan, greatly improve the work efficiency of dispatchers, and ensure the safe and stable operation of the power grid.

[0011] The present invention solves the technical problem by adopting the following technical solutions:

[0012] A development system based on intelligent IPC and efficient AI customization, including a basic process module, a media processing module, an algorithm processing module and a service maintenance module, wherein the basic process module is respectively connected to the media processing module, the algorithm processing module and the service maintenance module, the media processing module is connected to the algorithm processing module and the service maintenance module, and the algorithm processing module is connected to the service maintenance module. The basic process module is used to provide core business services and functions, including protocol processing, parameter setting, ball camera control, linkage processing, alarm processing, real-time recording and watchdog maintenance; the media processing module is used for basic video services, including video stream data processing, snapshot data service, motion detection service and occlusion alarm service; the algorithm processing module is used to provide algorithm docking services, standardize algorithm docking interfaces and processes, and realize fast and standardized access to third-party algorithms; the service maintenance module is used to maintain service communications between modules and provide a stable and reliable communication mechanism.

[0013] Moreover, it also includes an additional processing module, which is respectively connected to the basic process module, the algorithm processing module and the service maintenance module, and is used for performing third-party algorithm processing.

[0014] A processing method for a development system based on intelligent IPC efficient AI customization includes the following steps:

[0015] Step 1: Introduce AI open framework;

[0016] Step 2: Standardize services based on the AI ​​open framework;

[0017] Step 3: Standardize communications based on the AI ​​open framework;

[0018] Step 4: Provide external encapsulation frame delivery service based on service standardization and communication standardization;

[0019] Step 5: Output the document based on steps 1 to 4.

[0020] Moreover, the specific implementation method of step 1 is: introducing the concept of AI open framework. AI open framework is a communication method based on the communication library. Through the reasonable allocation and utilization of the platform's own resources, it fully releases the computing power of CPU, memory, NPU, basic hardware modules, etc. required by the algorithm. On the camera with computing power, it provides a standard algorithm and an integration method of algorithm application for various integrators and algorithm manufacturers, so that manufacturers without hardware industrialization capabilities can quickly implant algorithms into cameras as long as they have software algorithm development capabilities.

[0021] Moreover, the specific implementation method of step 2 is: combining the existing architecture and modules of traditional security audio and video, encapsulating methods, data, status, etc. into the form of services, and customers request the required services through standardized APIs, thereby realizing AI devices customized by customers.

[0022] Moreover, the specific implementation method of step 3 is: standardize the communication standard interface between each process and between different services and clients, provide a set of standard communication interface definitions and usage methods, including service registration, deregistration, request, release and data interface, to realize the basic operation of the service and data sending and receiving.

[0023] Moreover, the specific implementation method of step 4 is: in response to the third-party algorithm customization requirements, the original frame is obtained and processed accordingly. The open framework itself provides a Send Frame service to meet the most basic video source acquisition requirements of the algorithm process, and provides a set of standard APIs, as well as instruction documents including demonstration demos.

[0024] The advantages and positive effects of the present invention are:

[0025] 1. The present invention is redesigned on the basis of traditional IPC, optimizing the basic process, media processing process and algorithm processing process, and adding additional processing processes. The basic process and media processing process provide a variety of basic services, such as alarm service, email sending service, phone dialing service, alert service, snapshot service and video streaming service, etc., supporting third parties to develop functions according to specific needs; the algorithm processing process and additional processing process provide standard development rules, allowing third parties to replace self-developed processes to achieve algorithm iteration and optimization.

[0026] 2. The present invention can reduce the workload of custom development and adaptation and shorten the development cycle by introducing a development framework and open hardware resources.

[0027] 3. The present invention can significantly reduce the cost of system development and integration and improve development efficiency by opening up hardware resources and computing power and combining it with a development framework.

[0028] 4. The present invention can provide a standardized development framework and interface, so that the system can better adapt to new algorithms and application requirements, and improve the flexibility and scalability of the system. At the same time, through the standardized framework and interface, system maintenance and upgrades become simpler and more efficient, reducing the complexity of maintenance.

[0029] 5. The present invention combines algorithm manufacturers and hardware manufacturers better through a unified development framework, thus accelerating the development and market launch of intelligent cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a system structure diagram of the present invention;

[0031] Figure 2 A flowchart of the camera algorithm deployment steps of the present invention;

[0032] Figure 3 The present invention builds an algorithm processing flow chart based on the frame sending service;

[0033] Figure 4 Schematic diagram of relevant functions of the API of the present invention. DETAILED DESCRIPTION

[0034] The present invention is further described in detail below with reference to the accompanying drawings.

[0035] A development system based on intelligent IPC and efficient AI customization, such as Figure 1 As shown, it includes a basic process module, a media processing module, an algorithm processing module and a service maintenance module, wherein the basic process module is respectively connected to the media processing module, the algorithm processing module and the service maintenance module, the media processing module is connected to the algorithm processing module and the service maintenance module, and the algorithm processing module is connected to the service maintenance module. The basic process module is used to provide core business services and functions, including protocol processing, parameter setting, ball camera control, linkage processing, alarm processing, real-time recording and watchdog maintenance; the media processing module is used for basic video services, including video stream data processing, snapshot data service, motion detection service and occlusion alarm service; the algorithm processing module is used to provide algorithm docking services, standardize algorithm docking interfaces and processes, and realize fast and standardized access to third-party algorithms; the service maintenance module is used to maintain service communications between modules and provide a stable and reliable communication mechanism.

[0036] Meanwhile, the present invention further comprises an additional processing module, which is respectively connected to the basic process module, the algorithm processing module and the service maintenance module, and is used for performing third-party algorithm processing.

[0037] The focus of the system of the present invention is to encapsulate the traditional audio, video and alarm parts into services and solidify them into the program. The hardware resources such as memory, flash, computing power, etc. are opened up and can be used for the design and development of customized algorithms. In the two processes on the left side of the figure, many basic services are provided, including data services (real-time code stream, original frame, video code stream, intercom), method services (picture capture, peripheral control), status services (alarm status, system information), etc. These services can be called through the communication framework to realize the rapid implementation of intelligence.

[0038] like Figure 2 As shown, it is the core process of the third-party manufacturer algorithm and application development of the present invention. According to the process description, the industry-customized algorithm can be applied to the camera hardware to achieve rapid intelligence. Based on the communication library, the framework uses platform resources to effectively release the computing power of CPU, memory, NPU, etc., and provides standardized integration methods for various integrators and algorithm manufacturers, so that manufacturers without hardware industrialization capabilities can quickly embed software algorithms into cameras. In terms of service standardization, the traditional security audio and video architecture and modules are encapsulated into services, and customers can request customized AI devices through standardized APIs. Data services include frame delivery services (providing video frame preprocessing), real-time code streams (supporting multi-channel video / audio data streams), video code streams (storage devices containing videos and pictures), and voice intercoms (audio data transmission). Method services provide alarm processing (standard and custom alarm types and linkage processing) and picture capture (Jpeg encoding of specified or non-specified frames). The status service provides system information (recording, network, resource occupancy, etc.). Communication standardization standardizes the interface between processes and between services and clients, and provides standard communication interface definitions such as service registration, deregistration, request, release, and data interface. The open framework meets the needs of the algorithm process to obtain video sources through frame delivery services, and provides standard APIs and usage documentation, including demonstration demos, to ensure the platform's ease of use and standardization, reduce user development difficulty, and improve development and debugging efficiency.

[0039] A processing method for a development system based on intelligent IPC efficient AI customization includes the following steps:

[0040] Step 1: Introduce AI open framework.

[0041] The specific implementation method of step 1 is: introduce the concept of AI open framework. AI open framework is a communication method based on the communication library. Through the reasonable allocation and utilization of the platform's own resources, it fully releases the computing power of CPU, memory, NPU, basic hardware modules, etc. required by the algorithm. On the camera with computing power, it provides a standard algorithm and an integration method of algorithm application for various integrators and algorithm manufacturers, so that manufacturers without hardware industrialization capabilities can quickly implant algorithms into cameras as long as they have software algorithm development capabilities.

[0042] Step 2: Standardize services based on the AI ​​open framework.

[0043] The specific implementation method of step 2 is: combining the existing architecture and modules of traditional security audio and video, encapsulating methods, data, status, etc. into the form of services, and customers request the required services through standardized APIs, thereby realizing AI devices customized by customers.

[0044] The data services include:

[0045] Send Frame (send frame service), the video frame is in YUV or RGB format, and the original frame can undergo certain pre-processing (scaling, format conversion, frame rate control, etc.) before sending to the algorithm.

[0046] Live Sream (real-time code stream), supported by front-end devices, is a real-time encoded video / audio data stream that supports multi-channel and multi-stream access.

[0047] Record Stream (video code stream), supported by devices with storage, including videos and pictures, can provide query methods with multiple conditions.

[0048] Talk (voice intercom), supports the initial transmission of intercom audio data.

[0049] Method Service:

[0050] Alarm Process provides standard alarm types and customizable alarm types, and standard linkage processing methods. Customers can trigger alarms by themselves.

[0051] Snapshot (picture capture), provides Jpeg encoding of specified frames or non-specified frames.

[0052] Status Service:

[0053] System Info provides system information such as recording, network, and resource usage.

[0054] Step 3: Standardize communications based on the AI ​​open framework.

[0055] The specific implementation method of step 3 is: standardize the communication standard interface between each process and between different services and clients, provide a set of standard communication interface definitions and usage methods, including service registration, deregistration, request, release and data interface, to realize the basic operation of the service and data sending and receiving.

[0056] Step 4: Provide external encapsulation frame delivery service based on service standardization and communication standardization.

[0057] The specific implementation method of step 4 is: to obtain the original frame and perform corresponding processing according to the third-party algorithm customization requirements, the open framework itself provides the Send Frame service to meet the most basic video source acquisition requirements of the algorithm process, and also provides a set of standard APIs, as well as instruction documents including demonstration demos.

[0058] Step 5: Output the document based on steps 1 to 4.

[0059] According to the above-mentioned development system and processing method based on intelligent IPC efficient AI customization, the advantages and effects of the present invention are explained:

[0060] S1: Algorithm process construction (svp create).

[0061] Based on the hardware development environment and the cross-compilation chain provided, build an independent algorithm running process, compile the communication library and frame sending service together, and then the interface related calls can be realized. First understand the relevant interface description of the frame service, refer to the implementation in the sample code: test_main.c, and follow the steps below to realize video frame acquisition:

[0062] Save all .h header files in the frame_service.rar compressed package to the local code path and add the path to the corresponding Makefile;

[0063] Copy the dependent .so library to the local code path and configure it in the corresponding Makefile;

[0064] Write the code for getting the video frame locally, refer to the implementation of the sample, and copy the result away. Remember to notify the release of the frame in time after taking it away.

[0065] S2: Build the algorithm processing flow based on the frame delivery service.

[0066] like Figure 3 As shown, the present invention comprises the following steps:

[0067] 1. Initialize frame sending service

[0068] Initialize frame sending service

[0069] Function description: Initialize frame sending service

[0070] Interface definition: FC_Init(int_bufNum,const Size*_pstAttr)

[0071] Implementation method: Rationally evaluate the size and number of raw frame buffers that need to be initialized based on the algorithm business needs and hardware development resources. Ensure that they meet the hardware resource requirements and the memory required for algorithm operation.

[0072] 2. Start the frame delivery service

[0073] Function description: Register the frame sending service callback function

[0074] Interface definition: FC_Start(void)

[0075] Implementation method: Call this interface when you really need to obtain the original frame, in order to provide the possibility of pausing and starting the algorithm service, switching parameters, switching frame rates, etc. If there is no requirement for parameter update, you can actually call the startup interface to run the frame delivery service after initialization.

[0076] 3. Get the original frame (get_frame).

[0077] Function description: Able to obtain the original image address and related image information of the specified channel through the interface, including resolution, image format, etc.

[0078] Interface definition: FC_GetFrame(int_iChn, Comm_Frame*_pFrame, unsign int timeout)

[0079] Implementation method: The device channel number of a single channel is 0 by default, and the timeout unit is ms. You can also use the default blocking mode to pass -1. Determine whether the image is successfully acquired based on the actual interface return code.

[0080] 4. Send to algorithm processing (alg_process).

[0081] After obtaining the original frame, you can call the initialized algorithm module processing function, send the video image to the intelligent engine for processing, and obtain the algorithm analysis result cache.

[0082] 5. Release the original frame (release_frame).

[0083] Function description: Release the specified frame and corresponding storage information of the specified channel

[0084] Interface definition: FC_RelFrame(int_iChn, Comm_Frame*_pFrame)

[0085] Implementation method: Generally used in pairs with FC_GetFrame.

[0086] 6. Stop service

[0087] Function description: Pause the frame sending service and stop acquiring actions at the bottom layer with the original frame.

[0088] Interface Definition: FC_Stop(void)

[0089] Implementation method: Generally used in pairs with FC_Start.

[0090] S3: Service standardization.

[0091] The AI ​​open framework itself provides some basic services. In order to better meet customers' demands for AI customization, the framework provides a set of standardized service interfaces, which can open these services to third parties to achieve higher-quality intelligent applications.

[0092] The service-related methods provided by the core include service registration, deregistration, request, release and data interface.

[0093] Through Figure 4 The related APIs shown can implement the calling of basic services.

[0094] According to the above description, the present invention can greatly promote the development of algorithm application ecology, promote the widespread application and innovation of artificial intelligence technology in various industries, accelerate the process of intelligentization of various industries, and provide efficient solutions for the intelligentization of traditional industries such as electricity, education, railways, and water conservancy.

[0095] At the same time, the present invention provides a standard development model and interface standard, which solves the problems of inconsistent existing development models and poor compatibility, and provides a standard development environment and development method for efficient AI customization of hardware.

[0096] Finally, the present invention continuously optimizes the above development process based on the open framework, greatly improving the efficiency and flexibility of the intelligent camera hardware and greatly reducing the development and deployment costs. The present invention clarifies the development specifications and standards, realizes the rapid application of AI technology in different industry business fields, and promotes the rapid development of various industry ecosystems, which has important application value and economic significance.

[0097] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation manner. Any other implementation manners derived by those skilled in the art based on the technical solution of the present invention also fall within the scope of protection of the present invention.

Claims

1. A development system based on intelligent IPC and efficient AI customization, characterized by: It includes basic process module, media processing module, algorithm processing module and service maintenance module, among which the basic process module is respectively connected to the media processing module, algorithm processing module and service maintenance module, the media processing module is connected to the algorithm processing module and service maintenance module, and the algorithm processing module is connected to the service maintenance module. The basic process module is used to provide core business services and functions, including protocol processing, parameter setting, ball camera control, linkage processing, alarm processing, real-time recording and watchdog maintenance; the media processing module is used for basic video services, including video stream data processing, snapshot data service, motion detection service and occlusion alarm service; the algorithm processing module is used to provide algorithm docking service, standardize algorithm docking interface and process, and realize fast and standardized access of third-party algorithms; the service maintenance module is used to maintain service communication between modules and provide a stable and reliable communication mechanism.

2. According to claim 1, a development system based on intelligent IPC and efficient AI customization is characterized in that: It also includes an additional processing module, which is respectively connected to the basic process module, the algorithm processing module and the service maintenance module, and is used for performing third-party algorithm processing.

3. A processing method for a development system based on intelligent IPC efficient AI customization according to any one of claims 1 to 2, characterized in that: The following steps are involved: Step 1: Introduce AI open framework; Step 2: Standardize services based on the AI ​​open framework; Step 3: Standardize communications based on the AI ​​open framework; Step 4: Provide external encapsulation frame delivery service based on service standardization and communication standardization; Step 5: Output the document based on steps 1 to 4.

4. According to claim 3, a processing method for a development system based on intelligent IPC efficient AI customization is characterized in that: The specific implementation method of step 1 is: introduce the concept of AI open framework. AI open framework is a communication method based on the communication library. Through the reasonable allocation and utilization of the platform's own resources, it fully releases the computing power of CPU, memory, NPU, basic hardware modules, etc. required by the algorithm. On the camera with computing power, it provides a standard algorithm and an integration method of algorithm application for various integrators and algorithm manufacturers, so that manufacturers without hardware industrialization capabilities can quickly implant algorithms into cameras as long as they have software algorithm development capabilities.

5. According to claim 3, a processing method for a development system based on intelligent IPC efficient AI customization is characterized in that: The specific implementation method of step 2 is: combining the existing architecture and modules of traditional security audio and video, encapsulating methods, data, status, etc. into the form of services, and customers request the required services through standardized APIs, thereby realizing AI devices customized by customers.

6. According to claim 3, a processing method for a development system based on intelligent IPC efficient AI customization is characterized in that: The specific implementation method of step 3 is: standardize the communication standard interface between each process and between different services and clients, provide a set of standard communication interface definitions and usage methods, including service registration, deregistration, request, release and data interface, to realize the basic operation of the service and data sending and receiving.

7. According to claim 3, a processing method for a development system based on intelligent IPC efficient AI customization is characterized in that: The specific implementation method of step 4 is: in response to the third-party algorithm customization requirements, the original frame is obtained and processed accordingly. The open framework itself provides a Send Frame service to meet the most basic video source acquisition requirements of the algorithm process, and also provides a set of standard APIs and usage documentation including a demonstration demo.

Images