Method, device and system for realizing camera end side AI algorithm updating and camera

Abstract

The invention relates to the technical field of cameras, discloses an implementation method, device and system for updating an end-side AI algorithm of a camera and the camera, and aims to solve the problems that equipment resources are wasted and user requirements are difficult to meet in an existing implementation mode according to the end-side AI algorithm of the camera. The scheme mainly comprises the following steps: uploading an algorithm packet corresponding to a camera end side AI algorithm to a server, wherein the algorithm packet deploys a directory structure in the server; the mobile terminal sends a query request to the server according to the chip brand and the chip model of the bound camera, and the server queries the algorithm package file in the corresponding directory structure according to the query request and returns a query result to the mobile terminal; and the mobile terminal sends an algorithm downloading instruction to the camera according to the query result, and the camera downloads and installs a corresponding algorithm package file from the server according to the algorithm downloading instruction. According to the invention, hardware resources are saved, user requirements are met, and the method is suitable for a camera with an end-side AI function.

Description

Technical Field

[0001] The present application relates to the field of camera technology, and in particular to a method, device, system and camera for implementing camera-side AI algorithm update. Background Art

[0002] With the rapid development of artificial intelligence technology, cameras, as important terminal devices for applying AI technology, have gradually become the key to market competitiveness in terms of whether they have end-side AI functions. The end-side AI function of cameras refers to the direct integration of artificial intelligence technology into camera devices, enabling them to perform intelligent processing tasks on local devices without relying on cloud servers. The end-side AI function is implemented through end-side AI algorithms, which refer to artificial intelligence algorithms that run and process directly on the camera, allowing the camera to process data locally without sending the data to the cloud or server for processing. The main features of this algorithm include low latency, high efficiency, and powerful computing power, and it can reduce dependence on cloud resources and improve data security. It is suitable for various special scene requirements, such as target recognition, target tracking, intelligent early warning, and intelligent swarms.

[0003] However, since different users have different requirements for the AI ​​functions on the camera side, different users may need to install different AI algorithms on the camera side. In order to meet the needs of different users, the commonly used solution is to install multiple AI algorithms with different functions on the camera at the same time. However, the computing power and storage resources of the camera's neural network processing unit are limited, and users may only use the AI ​​algorithm on the camera side for a certain function, resulting in a waste of equipment resources. In practical applications, it is impossible to pre-install all possible AI algorithms on the camera when it leaves the factory, making it difficult to meet the needs of all users. Summary of the invention

[0004] The present application aims to solve the problems of waste of equipment resources and difficulty in meeting user needs in the existing camera-side AI algorithm implementation method, and proposes a camera-side AI algorithm update implementation method, device, system and camera.

[0005] The technical solution adopted by this application to solve the above technical problems is:

[0006] In a first aspect, the present application provides a method for implementing an AI algorithm update on a camera end, the method comprising:

[0007] Upload the algorithm package corresponding to the camera-side AI algorithm to the server. The algorithm package deploys a directory structure on the server. The directory structure includes the camera's chip brand directory, chip model directory, and algorithm package file directory in sequence. Multiple algorithm package files in the algorithm package are placed in the corresponding algorithm package file directory.

[0008] The mobile terminal sends a query request to the server according to the chip brand and chip model of the bound camera, and the server searches for the algorithm package file in the corresponding directory structure according to the query request and returns the query result to the mobile terminal;

[0009] The mobile terminal sends an algorithm download instruction to the camera according to the query result, and the camera downloads and installs the corresponding algorithm package file from the server according to the algorithm download instruction.

[0010] Furthermore, the algorithm package file includes an algorithm model file and an algorithm description file, wherein the algorithm model file is used to store and represent the algorithm model, and the algorithm description file is used to describe the algorithm model.

[0011] Furthermore, the algorithm description file is used to describe the algorithm model in JSON data format.

[0012] Furthermore, the camera downloads and installs the corresponding algorithm package file from the server according to the algorithm update instruction, specifically including:

[0013] After receiving the algorithm download instruction, the camera downloads the corresponding algorithm package file from the corresponding directory structure of the server. After downloading, the algorithm package file is decompressed and the legality and correctness of the algorithm model are verified. If the verification passes, the algorithm package file is installed in the preset directory.

[0014] Furthermore, the method further comprises:

[0015] The mobile terminal obtains the algorithm package file installed by the bound camera, and sends an algorithm uninstallation instruction to the camera according to the installed algorithm package file. The camera uninstalls the corresponding algorithm package file according to the algorithm uninstallation instruction.

[0016] Furthermore, the method further comprises:

[0017] After the camera installs the algorithm package file, it uses the AI ​​model framework to load and run the algorithm model corresponding to the algorithm package file, and sends the detection results during the algorithm model operation to the mobile terminal.

[0018] Furthermore, the AI ​​model framework includes a chip NPU SDK, an NPU function abstraction layer, an AI function layer and an AI cloud integrated application. The NPU function abstraction layer is used to automatically check the installed algorithm package files, and load and run the corresponding algorithm model. The AI ​​function layer is used to convert the detection data during the operation of the algorithm model into attribute data information described in the algorithm package file. The AI ​​cloud integrated application is used to upload the corresponding attribute data information to the AI ​​cloud, and the AI ​​cloud sends the detection results corresponding to the attribute data information to the mobile terminal.

[0019] In a second aspect, the present application provides a device for implementing an AI algorithm update on a camera side, wherein the device is used to implement the steps of the method for implementing an AI algorithm update on a camera side as described in the first aspect.

[0020] In a third aspect, the present application provides a system for implementing an AI algorithm update on a camera end, the system comprising:

[0021] The server is used to receive an algorithm package corresponding to an uploaded camera-side AI algorithm, and deploy a directory structure according to the algorithm package, wherein the directory structure includes a chip brand directory, a chip model directory, and an algorithm package file directory of the camera in sequence, and multiple algorithm package files in the algorithm package are placed in a corresponding algorithm package file directory; and query the algorithm package file in the corresponding directory structure according to a query request sent by the mobile terminal, and return the query result to the mobile terminal;

[0022] The mobile terminal is used to send a query request to the server according to the chip brand and chip model of the bound camera, and receive the query result sent by the server; and send an algorithm download instruction to the camera according to the query result, and the camera downloads and installs the corresponding algorithm package file from the server according to the algorithm download instruction.

[0023] In a fourth aspect, the present application provides a camera, the camera being used to receive an algorithm download instruction sent by a mobile terminal, and download and install a corresponding algorithm package file from a server according to the algorithm download instruction;

[0024] The algorithm download instruction is that the mobile terminal sends a query request to the server according to the chip brand and chip model of the bound camera, the server searches for the algorithm package file in the corresponding directory structure according to the query request, and returns the query result to the mobile terminal, and the mobile terminal sends the algorithm download instruction to the camera according to the query result;

[0025] The algorithm package file is an algorithm package file in the algorithm package corresponding to the camera-side AI algorithm transferred to the server. The algorithm package is deployed in a directory structure on the server. The directory structure includes the camera's chip brand directory, chip model directory and algorithm package file directory in sequence. Multiple algorithm package files in the algorithm package are placed in the corresponding algorithm package file directory.

[0026] The beneficial effects of the present application are: the implementation method, device, system and camera for updating the camera-side AI algorithm provided by the present application, the camera provides an update function for the end-side AI algorithm, allowing users to selectively update and install the end-side AI algorithm from the server according to their needs through the mobile terminal, thereby avoiding wasting equipment resources and meeting the needs of different users. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flowchart of a method for implementing a camera-side AI algorithm update provided in an embodiment of the present application;

[0028] Figure 2 A flowchart of another method for implementing an AI algorithm update on a camera end provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of the structure of another camera-side AI algorithm update implementation system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0031] In some of the processes described in the specification of this application and the above-mentioned figures, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The sequence numbers of the operations are only used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel.

[0032] The technical solution of the embodiments of the present application is applicable to cameras with end-side AI functions, such as smart cameras used for intelligent monitoring and intelligent security.

[0033] In order to meet the needs of different users, the commonly used solution for cameras with end-side AI functions is to install multiple end-side AI algorithms with different functions in the camera at the same time. The inventors have found through research that this solution not only wastes equipment resources, but also fails to meet the needs of all users.

[0034] Based on this, a solution of an embodiment of the present application is proposed. In the embodiment of the present application, an algorithm package corresponding to the camera-side AI algorithm is uploaded to a server, and a directory structure of the algorithm package is deployed on the server. The directory structure includes a chip brand directory, a chip model directory, and an algorithm package file directory of the camera in sequence, and multiple algorithm package files in the algorithm package are placed in the corresponding algorithm package file directory; the mobile terminal sends a query request to the server according to the chip brand and chip model of the bound camera, and the server queries the algorithm package file in the corresponding directory structure according to the query request, and returns the query result to the mobile terminal; the mobile terminal sends an algorithm download instruction to the camera according to the query result, and the camera downloads and installs the corresponding algorithm package file from the server according to the algorithm download instruction.

[0035] Specifically, the embodiments of the present application provide the camera with an update function of the end-side AI algorithm. By uploading the algorithm package corresponding to the camera end-side AI algorithm to the server, the user can query the algorithm file package that can be installed by the bound camera in the server through the mobile terminal, and can control the camera through the mobile terminal to download and install the specified algorithm package file, thereby meeting the user's demand for the camera end-side AI function, and there is no need to install algorithm file packages of multiple different functions when the camera leaves the factory, thereby saving the hardware resources of the camera.

[0036] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0037] Figure 1 A flowchart showing a method for implementing a camera-side AI algorithm update is shown in FIG. Figure 1 , the method comprises the following steps:

[0038] S1. Upload the algorithm package corresponding to the camera-side AI algorithm to the server. The algorithm package is deployed in a directory structure on the server. The directory structure includes the camera's chip brand directory, chip model directory, and algorithm package file directory in sequence. Multiple algorithm package files in the algorithm package are placed in the corresponding algorithm package file directory.

[0039] In an embodiment of the present application, the algorithm package file includes an algorithm model file and an algorithm description file, the algorithm model file is used to store and represent the algorithm model, and the algorithm description file is used to describe the algorithm model.

[0040] Among them, the algorithm model file refers to the file that stores and represents the algorithm model, usually including the parameters, structure and configuration information of the model. The algorithm description file is a document or code file used to record and describe the algorithm in detail, usually including the algorithm's logic, input and output requirements, parameter settings, and any necessary calculation steps.

[0041] In an embodiment of the present application, the algorithm description file uses the JSON data format to describe the algorithm model. JSON uses key-value pairs to represent data structures. The structure is simple and easy to read and write. JSON is a language-independent data format that can be read and generated by various programming languages.

[0042] After the algorithm package is uploaded to the server, a directory structure is deployed on the algorithm package server. The directory structure is divided into three levels, namely the camera chip brand directory, chip model directory, and algorithm package file directory. Multiple algorithm package files in the algorithm package are placed in the corresponding algorithm package file directory.

[0043] S2. The mobile terminal sends a query request to the server according to the chip brand and chip model of the bound camera. The server searches for the algorithm package file in the corresponding directory structure according to the query request and returns the query result to the mobile terminal.

[0044] The mobile terminal may be, but is not limited to, a smart phone, a personal computer, a tablet computer, etc.

[0045] In actual applications, the server provides a unified API interface for querying and downloading algorithm package files. Users can use the API interface to send query requests through mobile terminals, and then the server will feedback the query results to the mobile terminal. The query results are the list of installable algorithms bound to the camera. Users can choose to install the required algorithm package files based on the query results.

[0046] S3. The mobile terminal sends an algorithm download instruction to the camera according to the query result, and the camera downloads and installs the corresponding algorithm package file from the server according to the algorithm download instruction.

[0047] See also Figure 2 In actual applications, the user selects the algorithm to be installed in the installable algorithm list and sends the algorithm download instruction to the camera to trigger the algorithm download and installation process. After receiving the algorithm download instruction, the camera downloads the corresponding algorithm package file from the corresponding directory structure of the server. After the download is complete, the algorithm package file is decompressed and the legality and correctness of the algorithm model are verified. If the verification passes, the algorithm package file is installed in the preset directory.

[0048] In an embodiment of the present application, the method may further include: the mobile terminal obtains the algorithm package file installed by the bound camera, and sends an algorithm uninstallation instruction to the camera according to the installed algorithm package file, and the camera uninstalls the corresponding algorithm package file according to the algorithm uninstallation instruction.

[0049] In actual applications, after the mobile terminal establishes a network connection with the bound camera, the user can view the algorithm package files currently installed on the camera in the mobile terminal, and then uninstall unnecessary algorithm package files, thereby further saving the hardware resources of the camera.

[0050] See also Figure 2 In an embodiment of the present application, the method may further include: after the camera installs the algorithm package file, using the AI ​​model framework to load and run the algorithm model corresponding to the algorithm package file, and sending the detection results during the operation of the algorithm model to the mobile terminal.

[0051] In an embodiment of the present application, the AI ​​model framework includes a chip NPU SDK, an NPU function abstraction layer, an AI function layer and an AI cloud integrated application. The NPU function abstraction layer is used to automatically check the installed algorithm package file, and load and run the corresponding algorithm model. The AI ​​function layer is used to convert the detection data during the operation of the algorithm model into attribute data information described in the algorithm package file. The AI ​​cloud integrated application is used to upload the corresponding attribute data information to the AI ​​cloud. The AI ​​cloud sends the detection results corresponding to the attribute data information to the mobile terminal so that the user can view the detection results of the camera through the mobile terminal.

[0052] Among them, NPU SDK (Neural Processing Unit SDK) is a software development kit for cameras, specifically designed for developing and optimizing software running on the camera's NPU (neural processing unit). NPU SDK provides the necessary tools and libraries to help developers quickly build and deploy applications optimized for NPU.

[0053] To summarize, the implementation method of the camera-side AI algorithm update provided in the embodiment of the present application is that the camera provides an update function of the AI ​​algorithm on the end side. By uploading the algorithm package corresponding to the camera-side AI algorithm to the server, the user can query the algorithm file package that can be installed by the bound camera in the server through the mobile terminal, and can control the camera to download and install the specified algorithm package file through the mobile terminal, thereby meeting the user's demand for the camera-side AI function, and there is no need to install algorithm file packages of multiple different functions when the camera leaves the factory, thereby saving the hardware resources of the camera.

[0054] Based on the above technical solution, an embodiment of the present application also provides a device for implementing the update of the AI ​​algorithm on the camera side, and the device is used to implement the steps of the method for implementing the update of the AI ​​algorithm on the camera side described in the embodiment of the present application.

[0055] Based on the above technical solution, the embodiment of the present application also provides a camera end-side AI algorithm update implementation system, please refer to Figure 3 , the system comprising:

[0056] The server is used to receive an algorithm package corresponding to an uploaded camera-side AI algorithm, and deploy a directory structure according to the algorithm package, wherein the directory structure includes a chip brand directory, a chip model directory, and an algorithm package file directory of the camera in sequence, and multiple algorithm package files in the algorithm package are placed in a corresponding algorithm package file directory; and query the algorithm package file in the corresponding directory structure according to a query request sent by the mobile terminal, and return the query result to the mobile terminal;

[0057] The mobile terminal is used to send a query request to the server according to the chip brand and chip model of the bound camera, and receive the query result sent by the server; and send an algorithm download instruction to the camera according to the query result, and the camera downloads and installs the corresponding algorithm package file from the server according to the algorithm download instruction.

[0058] Based on the above technical solution, the embodiment of the present application further provides a camera, which is used to receive an algorithm download instruction sent by a mobile terminal, and download and install a corresponding algorithm package file from a server according to the algorithm download instruction;

[0059] The algorithm download instruction is that the mobile terminal sends a query request to the server according to the chip brand and chip model of the bound camera, the server searches for the algorithm package file in the corresponding directory structure according to the query request, and returns the query result to the mobile terminal, and the mobile terminal sends the algorithm download instruction to the camera according to the query result;

[0060] The algorithm package file is an algorithm package file in the algorithm package corresponding to the camera-side AI algorithm transferred to the server. The algorithm package is deployed in a directory structure on the server. The directory structure includes the camera's chip brand directory, chip model directory and algorithm package file directory in sequence. Multiple algorithm package files in the algorithm package are placed in the corresponding algorithm package file directory.

[0061] It can be understood that since the device, system and camera described in the embodiments of the present application are hardware platforms for implementing the method for updating the camera-side AI algorithm described in the embodiments, for the hardware platform disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple. For relevant matters, please refer to the partial description of the method, which will not be repeated here.

Claims

1. A method for implementing the camera-side AI algorithm update, characterized in that: The method comprises: Upload the algorithm package corresponding to the camera-side AI algorithm to the server. The algorithm package deploys a directory structure on the server. The directory structure includes the camera's chip brand directory, chip model directory, and algorithm package file directory in sequence. Multiple algorithm package files in the algorithm package are placed in the corresponding algorithm package file directory. The mobile terminal sends a query request to the server according to the chip brand and chip model of the bound camera, and the server searches for the algorithm package file in the corresponding directory structure according to the query request and returns the query result to the mobile terminal; The mobile terminal sends an algorithm download instruction to the camera according to the query result, and the camera downloads and installs the corresponding algorithm package file from the server according to the algorithm download instruction.

2. The method for implementing the camera end-side AI algorithm update according to claim 1, characterized in that: The algorithm package file includes an algorithm model file and an algorithm description file. The algorithm model file is used to store and represent the algorithm model, and the algorithm description file is used to describe the algorithm model.

3. The method for implementing the camera end-side AI algorithm update according to claim 2, characterized in that: The algorithm description file is used to describe the algorithm model in JSON data format.

4. The method for implementing the camera end-side AI algorithm update according to claim 1, characterized in that: The camera downloads and installs the corresponding algorithm package file from the server according to the algorithm update instruction, including: After receiving the algorithm download instruction, the camera downloads the corresponding algorithm package file from the corresponding directory structure of the server. After downloading, the algorithm package file is decompressed and the legality and correctness of the algorithm model are verified. If the verification passes, the algorithm package file is installed in the preset directory.

5. The method for implementing the camera end-side AI algorithm update according to claim 1, characterized in that: The method further comprises: The mobile terminal obtains the algorithm package file installed by the bound camera, and sends an algorithm uninstallation instruction to the camera according to the installed algorithm package file. The camera uninstalls the corresponding algorithm package file according to the algorithm uninstallation instruction.

6. The method for implementing the camera end-side AI algorithm update according to claim 1, characterized in that: The method further comprises: After the camera installs the algorithm package file, it uses the AI ​​model framework to load and run the algorithm model corresponding to the algorithm package file, and sends the detection results during the algorithm model operation to the mobile terminal.

7. The method for implementing the camera-side AI algorithm update according to claim 6, characterized in that: The AI ​​model framework includes a chip NPU SDK, an NPU function abstraction layer, an AI function layer and an AI cloud integrated application. The NPU function abstraction layer is used to automatically check the installed algorithm package file, and load and run the corresponding algorithm model. The AI ​​function layer is used to convert the detection data during the operation of the algorithm model into attribute data information described in the algorithm package file. The AI ​​cloud integrated application is used to upload the corresponding attribute data information to the AI ​​cloud, and the AI ​​cloud sends the detection results corresponding to the attribute data information to the mobile terminal.

8. A device for implementing the AI ​​algorithm update on the camera side, characterized in that: The device is used to implement the steps of the method for updating the camera-side AI algorithm as described in any one of claims 1 to 8.

9. A system for implementing the AI ​​algorithm update on the camera side, characterized in that: The system comprises: The server is used to receive an algorithm package corresponding to an uploaded camera-side AI algorithm, and deploy a directory structure according to the algorithm package, wherein the directory structure includes a chip brand directory, a chip model directory, and an algorithm package file directory of the camera in sequence, and multiple algorithm package files in the algorithm package are placed in a corresponding algorithm package file directory; and query the algorithm package file in the corresponding directory structure according to a query request sent by the mobile terminal, and return the query result to the mobile terminal; The mobile terminal is used to send a query request to the server according to the chip brand and chip model of the bound camera, and receive the query result sent by the server; and send an algorithm download instruction to the camera according to the query result, and the camera downloads and installs the corresponding algorithm package file from the server according to the algorithm download instruction.

10. A camera, characterized in that The camera is used to receive the algorithm download instruction sent by the mobile terminal, and download and install the corresponding algorithm package file from the server according to the algorithm download instruction; The algorithm download instruction is that the mobile terminal sends a query request to the server according to the chip brand and chip model of the bound camera, the server searches for the algorithm package file in the corresponding directory structure according to the query request, and returns the query result to the mobile terminal, and the mobile terminal sends the algorithm download instruction to the camera according to the query result; The algorithm package file is an algorithm package file in the algorithm package corresponding to the camera-side AI algorithm transferred to the server. The algorithm package is deployed in a directory structure on the server. The directory structure includes the camera's chip brand directory, chip model directory and algorithm package file directory in sequence. Multiple algorithm package files in the algorithm package are placed in the corresponding algorithm package file directory.

Images