Application provision device, application provision method, and program

Abstract

Provided are an application provision device, an application provision method, and a program capable of providing an application in a format suitable for a user device without increasing the burden of an application developer. The application providing device has an application conversion unit that converts the application acquired from the application database on the basis of device information acquired from the device information database as information relating to the device on which the application runs.

Description

Technical Field

[0001] The present technology relates to an application providing device, an application providing method, and a program. Background Art

[0002] An application providing system called an online market is available in which applications developed by developers, manufacturers, companies, individuals, etc. are sold and provided to users through the Internet (Patent Document 1).

[0003] In order for a user to deploy an application provided from an online market to a specific device and run the application, the application needs to be pre-converted into a format executable on the device according to differences in the device (hardware, operating system (OS) etc.).

[0004] Therefore, for a framework for developing applications for IoT devices, a technology has been proposed to package applications in advance and deploy the resulting package based on configuration information on a deployment target device (Patent Document 2).

[0005] Reference List

[0006] Patent Literature

[0007] Patent Document 1: U.S. Patent Publication No. 2013 / 0212160

[0008] Patent Document 2: U.S. Patent Publication No. 2018 / 0143825 Summary of the invention

[0009] Problems to be solved by the present invention

[0010] However, the technology disclosed in Patent Document 2 has the following disadvantages: pre-packaged applications increase the burden on application developers.

[0011] The present technology has been proposed in view of such circumstances, and therefore an object of the present technology is to provide an application providing device, an application providing method, and a program that allow applications to be provided in a format suitable for a user device without increasing a burden on application developers.

[0012] Solution to the problem

[0013] In order to solve the above-mentioned problem, a first technology is an application providing device including an application conversion unit that converts an application retrieved from an application database according to device information retrieved from the device information database as information about a device on which the application is executed.

[0014] Furthermore, a second technique is an application providing method including converting an application retrieved from an application database according to device information retrieved from the device information database as information about a device on which the application is executed.

[0015] Furthermore, a third technique is a program for causing a computer to execute an application providing method including converting an application retrieved from an application database according to device information retrieved from the device information database as information on a device on which the application is executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram showing the configuration of the application providing system 10 .

[0017] Figure 2 is a block diagram showing the configuration of the server device 500 .

[0018] Figure 3 is a block diagram showing the configuration of the application providing apparatus 100 .

[0019] Figure 4 This is an illustration of the three formats in WASM.

[0020] Figure 5 1 is a sequence diagram showing processes in the application providing system 10 and the application providing apparatus 100 .

[0021] Figure 6 1 is a sequence diagram showing processes in the application providing system 10 and the application providing apparatus 100 .

[0022] Figure 7 is a flowchart showing a process of switching the format of an application.

[0023] Figure 8 is a flowchart showing a process of switching the format of an application.

[0024] Fig. 9 is a diagram showing source code executed to compare processing times.

[0025] Fig.10 This is a table showing the comparison results of processing speeds.

[0026] Fig.11 is a diagram showing a first specific embodiment of the application providing system 10 .

[0027] Fig.12 is a diagram showing a second specific embodiment of the application providing system 10 .

[0028] Fig.13 is a block diagram showing the software configuration of the target device 400t.

[0029] Fig.14 400 is a diagram showing a first specific embodiment of a process performed under the software configuration of the target device 400t.

[0030] Fig.15400 is a diagram showing a second specific embodiment of the processing performed under the software configuration of the target device 400t.

[0031] Fig.16 is a diagram showing a GUI for registering the device 400 in the application providing device 100 .

[0032] Fig.17 is a diagram showing a GUI for registering the device 400 in the application providing device 100 .

[0033] Fig.18 is a simplified diagram showing a GUI for deploying an application to a target device 400t.

[0034] Fig.19 is a simplified diagram showing a GUI for deploying an application to a target device 400t.

[0035] Fig. 20 is a simplified diagram showing a GUI for deploying an application to a target device 400t.

[0036] Fig.21 is a diagram showing a GUI for undeploying an application. DETAILED DESCRIPTION

[0037] Hereinafter, embodiments of the present technology will be described with reference to the drawings. Note that the description will be given in the following order.

[0038] <1. Implementation Method>

[0039] [1-1. Configuration of application providing system 10]

[0040] [1-2. Configuration of application providing apparatus 100]

[0041] [1-3. Processing in Application Providing Device 100]

[0042] [1-4. Software Configuration of Target Device 400t]

[0043] [1-5. Description of GUI for device registration and application deployment]

[0044] <2. Modifications>

[0045] <1. Implementation Method>

[0046] [1-1. Configuration of application providing system 10]

[0047] First, refer to Figure 1The configuration of the application providing system 10 is described. The application providing system 10 includes an application providing device 100, a developer-side terminal device 200 used by an individual (referred to as a developer) who develops an application program (hereinafter, referred to as an application), a user-side terminal device 300 used by an individual (referred to as a user) who uses the application, and a device 400 on which the application is run. Note that the developer is not limited to an individual and may be a manufacturer, a company, etc. In addition, the user is also not limited to an individual and may be a manufacturer, a company, etc.

[0048] The developer-side terminal device 200, the user-side terminal device 300, and the device 400 are each connected to the application providing device 100 via a network. Figure 1 One developer-side terminal device 200 and one user-side terminal device 300 are shown, but in reality, a plurality of developer-side terminal devices 200 and a plurality of user-side terminal devices 300 are connected to the application providing device 100 through a network.

[0049] The application providing device 100 is responsible for registering and storing applications uploaded from the developer-side terminal device 200, and uses the user-side terminal device 300 to transfer and deploy the application requested by the user to the device 400 selected by the user for deployment. The device 400 selected by the user from multiple devices 400 to run the application is called the target device 400t. Note that the conditions that each application needs to meet to allow registration may include: the application has a memory model that does not depend on the target device 400t, and two or at least one of the following: the application has the ability to call a predetermined application programming interface (API), or the application itself has an API that receives a predetermined call. In addition, the conditions for allowing registration are not necessary, and registration can be allowed without imposing conditions.

[0050] By using the application providing apparatus 100, a so-called online market can be realized in which applications are provided online in response to user requests. Note that each application may be provided as a paid service based on a purchase requiring user payment, or may be provided free of charge without requiring monetary payment.

[0051] Each application developer uploads the application to the application providing apparatus 100 for registration using the developer-side terminal device 200. For example, the developer may develop the application using the developer-side terminal device 200, or may develop the application using a device other than the developer-side terminal device 200 and use the developer-side terminal device 200 to communicate with the application providing apparatus 100 for uploading the application. Embodiments of the developer-side terminal device 200 include a personal computer, a smart phone, a tablet terminal, a server device, etc. The developer-side terminal device 200 may be a system including a server provided with a development environment.

[0052] The user-side terminal device 300 is used by the user to request the application providing device 100 to provide an application, select a target device 400t on which to run an application, etc. Examples of the user-side terminal device 300 include a personal computer, a smart phone, a tablet terminal, a wearable device, and the like.

[0053] Embodiments of the device 400 include various devices and equipment capable of running an application requested by a user using the user-side terminal device 300. Specifically, embodiments of the device 400 include an Internet of Things (IoT) device, a personal computer, a server device, a tablet terminal, a smart phone, a smart watch, a wearable device, a camera, a portable game console, a home appliance, a mobile unit such as a car or a drone, etc. However, the device 400 may be any device or equipment capable of running an application. The device 400 includes a CPU, a memory, a storage medium, etc. that runs an application.

[0054] The user needs to connect the device 400 to the application providing device 100 through the network in advance. In addition, the device 400 may be pre-registered in the application providing device 400. The application providing device 100 may be connected to one or more devices 400. For example, there may be a case where the device 400 is a camera, a plurality of cameras are pre-connected to the application providing device 100, a specific camera is used to detect a person, and other cameras are used to detect an object, that is, the usage is different for each camera. In such a case, it is also possible to select an application and a camera and deploy a different application to each camera.

[0055] The user-side terminal device 300 and the device 400 may be identical. For example, when a user uses a personal computer as the user-side terminal device 300 and runs an application provided by the application providing device 100 on the personal computer, the user-side terminal device 300 and the device 400 are identical.

[0056] Before an application is registered in the application providing apparatus 100 , the developer-side terminal apparatus 200 and the application providing apparatus 100 need to perform mutual authentication (account authentication, server authentication), establish secure communication, verify the developer's user signature, and the like.

[0057] Furthermore, the user-side terminal device 300 and the application providing device 100 need to perform mutual authentication (device 400 authentication, server authentication), establish secure communication, verify application signatures, etc. before the user registers with the application providing device 100. Note that the application signature may be verified after application deployment.

[0058] In this embodiment, the application providing device 100 includes, for example, Figure 2 The server device 500 shown in FIG. 5 includes at least a control unit 501 , a storage unit 502 , and a communication unit 503 .

[0059] The control unit 501 includes a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), etc. The CPU controls the entire server device 500 and each unit by executing various processes according to a program stored in the ROM to issue a command.

[0060] The storage unit 502 is a large-capacity storage medium including, for example, a hard disk, a flash memory, etc. The storage unit 502 stores programs used for the operation of the application providing system 10, other data, and the like.

[0061] The communication unit 503 is a communication interface with the developer side terminal device 200, the user side terminal device 300, the device 400, the network, etc. The communication method can be wired or wireless. Examples of communication methods include cellular communication, 4G, 5G, Wi-Fi, Bluetooth (registered trademark), near field communication (NFC), Ethernet (registered trademark), High Definition Multimedia Interface (HDMI (registered trademark)), Universal Serial Bus (USB), etc. Note that it is preferred to encrypt the communication using Transport Layer Security (TLS) or Secure Sockets Layer (SSL).

[0062] The application providing system 10 is configured as described above.

[0063] [1-2. Configuration of application providing apparatus 100]

[0064] Next, we will refer to Figure 3 The configuration of the application providing apparatus 100 according to the present embodiment is described.

[0065] The application providing device 100 includes a developer and application management unit 101 , an application database 102 , a user and device management unit 103 , an application selection unit 104 , a device information database 105 , a device information selection unit 106 , an application conversion unit 107 and a providing-side deployment management unit 108 .

[0066] The developer and application management unit 101 obtains the application uploaded from the developer-side terminal device 200 and outputs the application to the application database 102 to register the application in the application providing device 100. In addition, the developer and application management unit 101 is responsible for registering the developer information transmitted from the developer-side terminal device 200, providing the development environment to the developer-side terminal device 200, managing the application-related transactions performed by the developer-side terminal device 200, etc.

[0067] For example, the developer information includes information such as the developer's name, login information, ID, password, application details, and application type.

[0068] The application database 102 is a database for storing applications uploaded from the developer-side terminal device 200. The applications stored in the application database 102 are in a platform-independent format. The platform-independent format is, for example, WebA assembly (WASM) bytecode. In the present embodiment, all applications uploaded from the developer-side terminal device 200 and stored in the application database 102 are in WASM. Note that the platform-independent format may be source code written in C++, Java (registered trademark), JavaScript (registered trademark), or other languages, or object code for a virtual machine. It is assumed that multiple applications uploaded and registered from one or more developer-side terminal devices 200 are stored in the application database 102. Each application can access the target device 400t only via an API provided by an entity such as a company (such as an online market) that provides services using the application providing device 100, or an API allowed by the company.

[0069] The user and device management unit 103 is responsible for registering information about the device 400 transmitted from the user-side terminal device 300 (referred to as device information), registering information about the user, managing transactions related to application requests and selection of the target device 400 t, and the like.

[0070] The user information is information about the user, such as the user's name, login information, ID, and password. The device information is, for example, information such as the type of the device 400, device identification information such as the serial number of the device 400, the certificate of the device 400, the configuration of the device 400, and the platform architecture of the device 400 as a deployment target necessary for application conversion.

[0071] The user and device management unit 103 outputs application identification information indicating the application requested by the user using the user side terminal device 300 to the application selection unit 104. Also, device identification information indicating the target device 400t selected by the user using the user side terminal device 300 is output to the device information selection unit 106.

[0072] The application selection unit 104 selects and retrieves an application requested by a user from the application database 102 based on the application identification information, and outputs the application to the application conversion unit 107 .

[0073] The device information database 105 is a database that stores device information.

[0074] The device information selection unit 106 selects and retrieves device information (referred to as target device information) about the target device 400 t selected by the user from the device information database 105 based on the device identification information, and outputs the device information to the application conversion unit 107 .

[0075] The application conversion unit 107 converts the application selected by the application selection unit 104 into an object file of the target device 400t based on the target device information. For example, if the application is in WASM format, the WASM bytecode is converted into an object code for the target device 400t. This application conversion is called compilation.

[0076] The application conversion unit 107 outputs the converted application to the supply-side deployment management unit 108. Note that the application may be converted during deployment to the target device 400t, or the application may be pre-converted into an object code for each platform and each object code may be stored in the application database 102.

[0077] In the present technology, application formats include WASM bytecode, Advance of Time (AoT), and Native. WASM bytecode is an instruction format that runs on a virtual machine and requires a library including an interpreter to run. Both AoT and Native are instruction formats that can run directly on the target device, but AoT includes code to check memory addresses accessed at runtime, and Native does not include such code. In addition, AoT requires a library for the above-mentioned memory address check performed at runtime, but Native does not need such a library.

[0078] WASM is a technology that enables the development of applications that can run in different runtime environments, especially on devices with limited computing power and memory size, such as IoT devices. However, an interpreter is required to run WASM, which makes devices with limited processing power face problems in terms of processing time and required memory. When an application is compiled into the AoT format that does not require an interpreter (i.e., compiled into AoT executable code), the application can run quickly and have less memory usage even on devices with limited processing power. In addition, for faster and more memory-efficient execution, the application can be converted into Native code with memory checks disabled.

[0079] The difference between Native and AoT is the presence or absence of a storage check code, and Native does not have a storage check code, but AoT has a storage check code. Therefore, the application converted to AoT has a memory check code. Native is faster in processing speed, but is allowed to access the memory of the target device 400t to which the application has been deployed, and therefore can access memory that should not be accessed, which causes security issues. When an application is converted, information for managing memory access permissions is added to the application. By adding information during AoT conversion, the memory access of the application can be managed.

[0080] There are three types of WASM execution methods. Figure 4The interpreter of the first method shown in A involves sequential compilation to Native line by line at runtime. Since the compilation is performed sequentially for each line, the processing takes more time, uses more memory, and has poorer performance than other formats. In addition, in order to execute, the target device 400t needs to have an interpreter.

[0081] Just-in-time (JIT) compilation (i.e. Figure 4 The second method shown in B) involves precompiling the code into an intermediate representation (IR) and compiling the IR into Native at runtime. The initial execution performance is poor, but the subsequent execution performance is close to Native. For JIT compilation, the target device 400t needs to have a JIT compiler.

[0082] As Figure 4 The AoT compilation of the third method shown in C involves precompiling the code into a Native customized for the hardware. The processing can be performed with execution performance close to (but not the same as) the Native. AoT compilation requires neither an interpreter nor a JIT compiler (but requires a runtime for memory checking). AoT compilation is recommended when the target device 400t has limited memory capacity. However, the present technology is not intended to exclude interpreters and JIT compilers, and can provide applications for conversion of interpreters or JIT compilation.

[0083] In addition, the application conversion unit 107 adds a verification signature to the application subjected to conversion. The signature is added to the binary hash of the application. The signature is used to verify the integrity of the application. By verifying the integrity, tampering can be detected, and therefore, security is ensured.

[0084] Compared to WASM, AoT allows for a reduction in processing time and memory usage, but allows access to the hardware and memory of the target device 400t, which leads to the risk of unauthorized access to the application. In order to avoid risks, it is necessary to take measures such as only providing applications developed by trusted developers, and not placing confidential information on the target device 400t when providing applications developed by untrusted developers. However, both measures limit the applications that can be provided and impair the convenience of developers and users. Therefore, only applications that have added a verification signature and whose integrity has been verified by verification of the signature before being deployed to the target device 400t can run on the target device 400t. However, the application providing device 100 can also provide applications without adding a signature.

[0085] return Figure 3. The provision-side deployment management unit 108 deploys the application converted by the application conversion unit 107 to the target device 400t. Deployment is the action of making the application operate on the target device 400t, and is completed when the application is downloaded to the target device 400t. Therefore, for the application, deployment, signature verification, and installation are performed in this order. In addition, the provision-side deployment management unit 108 manages deployment-related transactions with the target device 400t. Under the management of the application provision device 100, the deployment of the application to the target device 400t is performed. Anti-tampering measures including encryption and signing are taken for the target code of the target device 400t.

[0086] Note that the format of the application deployed to the target device 400t may be a platform-independent format, such as WASM bytecode, native code for the CPU of the target device 400t, AoT-compiled code (obtained by adding security check code to native code), and the like.

[0087] As described above, the application providing device 100 is configured. The developer and application management unit 101, the user and device management unit 103, the application selection unit 104, the device information selection unit 106, the application conversion unit 107, and the provision side deployment management unit 108 are implemented by the processing in the control unit 501. In addition, each processing unit can be implemented by a hardware processing block having a dedicated function. In addition, the application database 102 and the device information database 105 are configured by the storage unit 502.

[0088] In the present embodiment, the server device 500 is used as the application providing device 100, but the application providing device 100 and the application providing method can be implemented by a program executed by any type of electronic device equipped with a computer function (such as a personal computer, a tablet terminal, or a smart phone). The program may be pre-installed on the server device 500 or the electronic device, or may be distributed via download, storage medium, etc. and installed by a user, etc. In addition, the application providing device 100 may be configured as a single device. The application providing device 100 may be a system including a plurality of servers connected to each other via a network.

[0089] Note that Figure 3As shown, the developer-side terminal device 200 includes a developer-side management unit 201 provided to use the application providing device 100. The developer-side management unit 201 is responsible for uploading the application to the application providing device 100, transmitting the developer information to the application providing device 100, managing the transaction with the application providing device 100 based on the input operation of the developer, etc. In addition, the developer-side management unit 201 is also responsible for controlling a graphical user interface (GUI) for using the application providing device 100, etc. The developer-side management unit 201 can be implemented by a program executed by the developer-side terminal device 200, or can be implemented by a dedicated hardware processing block.

[0090] In addition, the user-side terminal device 300 includes a user-side management unit 301 provided to use the application providing device 100. The user-side management unit 301 is responsible for performing a process of transmitting device information to the application providing device 100, a process of requesting an application, a process of selecting a target device 400t based on a user's input operation, etc., and manages transactions performed with the application providing device 100. In addition, the user-side management unit 301 is also responsible for controlling a graphical user interface (GUI) for using the application providing device 100, etc. The application management unit may be implemented by a program executed by the user-side terminal device 300, or may be implemented by a dedicated hardware processing block.

[0091] In addition, the device 400 includes a device-side deployment management unit 401. The device-side deployment management unit 401 is responsible for executing the process of deploying the application to the device 400 under the management of the provision-side deployment management unit 108 of the application provision device 100. In addition, the device-side deployment management unit 401 is also responsible for managing deployment-related transactions performed by the application provision device 100. The device-side deployment management unit 401 may be implemented by a program executed by the device 400, or may be implemented by a dedicated hardware processing block.

[0092] [1-3. Processing in the Application Providing System 10 and the Application Providing Device 100]

[0093] Next, we will refer to Figure 5 and Figure 6 The processing in the application providing system 10 and the application providing apparatus 100 is described. Figure 5 and Figure 6 , for the sake of the drawing, the user and device management unit 103, the application selection unit 104, the application conversion unit 107, and the provision-side deployment management unit 108 are collectively shown.

[0094] First, we will refer to Figure 5The application storage phase is described. In step S11, the developer-side terminal device 200 transmits the developer information to the application providing device 100 based on the developer's input operation. Then, in step S12, the developer and the application management unit 101 register the developer information. The registration of the developer information can be performed by storing the developer information with the developer information associated with the application in the application database 102, or by providing a database for the developer information and storing the developer information in the database.

[0095] When the developer registration is completed, in step S13, the application providing apparatus 100 transmits the application development environment to the developer-side terminal apparatus 200, and in step S14, the developer-side terminal apparatus 200 acquires the development environment. Note that the development environment may be provided by the application providing apparatus 100, or may be provided by another apparatus.

[0096] Then, after the developer has completed the development of the application, in step S15, the developer-side terminal device 200 uploads the application to the application providing device 100 based on the developer's input operation. For example, the uploading of the application is performed by directly inputting the application in the ".wasm" format to the application providing device 100. Note that the verification process may be performed before the application is stored in the application providing device 100.

[0097] Next, in step S16 , the developer and application management unit 101 outputs the uploaded application to the application database 102 , and in step S17 , the application is stored in the application database 102 .

[0098] Next, we will refer to Figure 5 The application request phase is described. First, in step S21, the user-side terminal device 300 transmits user information to the application providing device 100 based on the user's input operation. Then, in step S22, the user and device management unit 103 registers the user information. The registration of the user information can be performed by storing the user information and the user information associated with the device information in the device information database 105, or by providing a database of the user information and storing the user information in the database.

[0099] Next, after registering the user information, in step S23, the user-side terminal device 300 transmits device information about the device 400 to the application providing device 100 based on the user's input operation. Then, in step S24, the user and device management unit 103 outputs the device information to the device information database 105. Then, in step S25, the registration of the device information is performed by storing the device information in the device information database 105. Note that the user can pre-register not only the target device 400t for running the application in the near future, but also all the devices 400 that are managed and used by the user and may run the application. Note that the transmission of the device information to the application providing device 100 to register the device 400 can be performed by a person other than the user.

[0100] After the user information and the device information are registered, as shown in step S26 , the user-side terminal device 300 requests the application providing device 100 to provide an application based on the user's input operation.

[0101] In order to request an application, information (name, developer name, function, etc.) about each application that can be provided by the application providing apparatus 100 (applications stored in the application database 102) is presented to the user, for example, through the GUI of the user-side terminal device 300. Then, when the user performs an input operation to select one of the applications, the user-side terminal device 300 transmits information indicating the selected application to the application providing apparatus 100.

[0102] Then, in step S27 , the application selection unit 104 retrieves the application requested by the user from the application database 102 based on the application identification information, and outputs the application to the application conversion unit 107 .

[0103] Next, we will refer to Figure 6 The application conversion and deployment phase is described. First, when the user selects the device 400 (target device 400t) to which the application is deployed, as shown in step S31, the user-side terminal device 300 transmits target device selection information to the application providing device 100.

[0104] For example, in order to select the target device 400t, information (name, serial number, etc.) for identifying the corresponding device 400 registered in the application providing device 100 is presented to the user through the GUI of the user-side terminal device 300. Then, when the user performs an input operation to select one of the devices 400, the user-side terminal device 300 transmits target device selection information indicating the selected device to the application providing device 100.

[0105] Then, the user and device management unit 103 outputs device identification information indicating the selected target device 400 t to the device information selection unit 106 .

[0106] Next, in step S32 , the device information selection unit 106 retrieves target device information from the device information database 105 based on the device identification information, and outputs the target device information to the application conversion unit 107 .

[0107] Note that when requesting an application, the user may select another target device 400t continuously, or may select another target device 400t after a period of time has passed since requesting the application. The request for the application and the selection of the device 400 do not have to be performed simultaneously or continuously within a predetermined period of time. In addition, the application may be requested after the target device 400t is selected.

[0108] Next, in step S33, the application conversion unit 107 converts the application requested by the user into an object file of the target device 400t based on the target device information. It is possible to identify on which device 400 the application is running based on the target device information, so that the application can be converted according to the target device 400t. Note that, although the details will be described later, the application conversion unit 107 can switch the format of the application subjected to conversion.

[0109] Next, in step S34 , the application conversion unit 107 adds a verification signature to the converted application. The application conversion unit 107 outputs the converted and signed application to the supply-side deployment management unit 108 .

[0110] Next, in step S35 , the provision-side deployment management unit 108 deploys the application to the target device 400 t .

[0111] Then, as shown in step S36, the signature is verified by the target device 400t. The public key can be used to verify the signature. The public key can be provided by, for example, the developer of the application or an individual who is the same as or related to the operator of the online market using the application providing device 100. Preferably, the public key is stored in the target device 400t before the application is provided. The public key can be provided together with a public key certificate issued by a trusted authority. In the event that the signature verification is successful, the application is installed on the target device 400t. This makes the application operable on the target device 400t, as shown in step S37.

[0112] On the other hand, in the event that the signature verification fails, the application is not installed on the target device 400t. Alternatively, the application may be installed but prevented from running.

[0113] Note that, in the event that unauthorized memory access by an application is detected as a result of memory check of the target device 400 t , error processing may be performed, although depending on how the target device 400 t is implemented.

[0114] Here, switching of the format of the application converted by the application conversion unit 107 will be described.

[0115] Figure 7 The first embodiment of switching formats is shown. Note that it is assumed that the application in the initial state is WASM bytecode.

[0116] First, in step S41 , the application conversion unit 107 acquires a configuration corresponding to the target device information. Note that the device information selection unit 106 retrieves the configuration from the device information database 105 and outputs the configuration to the application conversion unit 107 .

[0117] Then, in step S42 , the application conversion unit 107 switches the format of the application to AoT.

[0118] Note that the format of the application may be switched based on various information about the target device 400 t including not only the configuration of the target device 400 t but also the status, specifications, and the like of the hardware and software of the target device 400 t .

[0119] Figure 8 A second embodiment of switching the format of an application is shown. Note that it is assumed that the application in the initial state is WASM bytecode.

[0120] In a case where the processing speed of the application on the target device 400 t is prioritized in step S51 , the process proceeds to step S52 (YES in step S51 ).

[0121] Next, in a case where security measures are not required when the application is running on the target device 400 t in step S52 , the process proceeds to step S53 (YES in step S52 ).

[0122] Then, in step S53, the application conversion unit 107 switches the format of the application to Native. This is because Native is faster in processing speed than WASM or AoT, and when processing speed is prioritized and security measures are not required, it is preferable to reduce processing time by compiling to Native.

[0123] The case where security measures are not required is, for example, a case where the developer of the application is a trusted developer that is the same as or related to the operator of the online market using the application providing apparatus 100 and there is no security problem. This is because an application compiled as Native can access the hardware and memory of the target device 400t without restriction, but even if an application developed by such a trusted developer accesses the hardware and memory of the target device 400t, it is considered safe. Whether the developer of the application is the same as or related to the operator of the online store using the application providing apparatus 100 can be determined based on the developer information registered in the above step S12.

[0124] On the other hand, in a case where the processing speed of the application on the target device 400t is not prioritized in step S51 (No in step S51), or in a case where security measures are required in step S52, the process proceeds to step S54 (No in step S52).

[0125] Then, in step S54 , the application conversion unit 107 switches the format of the application to AoT.

[0126] Note that steps S51 and S52 may be performed in reverse order.

[0127] exist Figure 8 In the illustrated process, the format is switched based on the processing speed and the safety measure, but the format may be switched based only on whether the processing speed is prioritized, or the format may be switched based only on whether the safety measure is required.

[0128] Information indicating whether to prioritize the processing speed and whether security measures are required may be input by the user using the user-side terminal device 300 and input from the user and device management unit 103 to the application conversion unit 107. Alternatively, "the processing speed is prioritized for this application" may be predetermined as application-specific information. For example, the developer of the application is the same as or related to the operator of the online market using the application providing device 100, and the situation where there is no security problem automatically eliminates the need for security measures. In this case, the user does not have to input whether security measures are required.

[0129] Note that Figure 7 and Figure 8 In the illustrated process, the format of the application is switched to Native or AoT; however, in the case where it is preferred to switch the format to WASM bytecode, a corresponding determination process can be added to enable switching to WASM bytecode. For example, WASM bytecode can be used when the deployment target is a high-performance device 400, a high-performance cloud server, etc., and it is acceptable to run the application using an interpreter.

[0130] Apart from Figure 7 and Figure 8 In addition to the method illustrated in FIG. 4 , the format to which the conversion is applied may be switched according to the state and use of the target device 400 t obtained, for example, through feedback from the target device 400 t.

[0131] For example, in the case where the target device 400t does not perform a specific process, it is not necessary to prioritize the processing speed, so the application converted into AoT or WASM bytecode is deployed to the target device 400t. Thereafter, in the case where feedback indicating that it is necessary to perform the process at a high speed has been received from the target device 400t, for example, the same application is converted into Native and deployed.

[0132] Furthermore, for example, in a case where security measures are not required and the application is first converted to Native and deployed to the target device 400t, but feedback indicating that security measures are required is received while the target device 400t is in operation, the application is converted to AoT and deployed again to the device 400. For this purpose, the application providing device 100 only needs to output the feedback received from the target device 400t to the application conversion unit 107.

[0133] Note that the application may be converted into a format specified by the user. For example, the above operation may be realized by displaying a list of available formats on the GUI of the user-side terminal device 300 and allowing the user to select one of the formats and outputting the corresponding selection information to the application conversion unit 107.

[0134] Here, we show the comparison results of processing time when running the same source code converted to Native, WASM bytecode, and AoT. Fig. 9 Same as shown in .

[0135] Fig.10 A shows a microcontroller called ESP32 developed by Esppressif Systems running on Fig. 9 The microcontroller includes a Tensilica Xtensa microprocessor.

[0136] The processing time is 0.99 seconds for Native, 28.2 seconds for WASM bytecode, and 2.09 seconds for AoT.

[0137] Fig.10 B shows the operation of the system on a board computer called Spresence (registered trademark) developed by Sony Corporation and capable of incorporating a smart sensing processor for IoT applications. Fig. 9 The result of the source code shown in , which is converted to Native, WASM bytecode, and AoT.

[0138] The processing time is 0.99 seconds for Native, 20.2 seconds for WASM bytecode, and 2.01 seconds for AoT.

[0139] In either case, WASM bytecode has a longer processing time due to the inclusion of interpreter processing and other factors. In addition, AoT has a shorter processing time than WASM bytecode. Therefore, in cases where reducing processing time is a priority, it is preferable to compile to AoT rather than WASM bytecode.

[0140] However, at the assembly code level, the processing time of AoT is about twice as long as that of Native due to the protection processing for memory access management. Therefore, in cases where compiling to AoT is not necessary (for example, when the developer of the application is a trusted individual and there are no security issues, etc.), it is preferred to compile to Native.

[0141] Fig.11 and Fig.12 is a diagram showing a specific example of the application providing system 10. Actually, the application providing system 10 can be configured in this way. Fig.11 and Fig.12 In the specific embodiment described in , the target device 400t is a camera. In addition, the developer may upload the application to the application providing device 100 using an online source code management service such as GitHub (registered trademark) or Bitbucket (registered trademark).

[0142] [1-4. Software Configuration of Target Device 400t]

[0143] Next, the software configuration of the target device 400t on which the application is executed will be described. Fig.13 As shown, the software configuration includes a pre-trained AI model 601, a signal processing application 602, a recognition application 603, an application framework 604, and a user application 605.

[0144] In the present technology, the identification application 603 is an application provided by the application providing apparatus 100. However, Fig.13 Any application or AI model shown in the figure may be provided by the application providing device 100, or only some applications or AI models may be provided by the application providing device 100. In addition, the functions of each application and AI model may be partially or completely implemented by hardware.

[0145] The application framework 604 allows the pre-trained AI model 601, the signal processing application 602, and the recognition application 603 to exchange data and other information with the user application 605 via their respective predetermined APIs. For example, APIs such as Initialize, Analyze, Resultrelease, Finalie, GetPPLversion, and AddLicense are defined between the application framework 604 and the recognition application 603.

[0146] Initialize is used to initialize the application. Use Analysis to perform a process of analyzing the inference results from the AI ​​model. Result Release is used to release the memory allocated in the recognition application 603 during the analysis process. Finalize is used to perform a process of terminating the recognition application 603. GetPplVersion is used to obtain the version information of the application. The application developed by the developer, the application stored in the application providing device 100, and the application deployed to the target device 400t may each include a processing unit corresponding to at least one or all of the APIs.

[0147] Next, a specific example of processing performed under the software configuration of the target device 400 t will be described.

[0148] Fig.14 The first specific embodiment described in is the classification of subjects appearing in an image. Fig.14 In the illustrated embodiment, the object appearing in the captured image is a car. First, a pre-trained AI model 601 performs inference on the image captured by the camera and outputs the inference results to a signal processing application 602 before formatting.

[0149] Next, the signal processing application 602 calculates a score for each category in which the object recognized in the captured image is classified as the formatted inference result, and outputs the score to the recognition application 603. Here, it is assumed that a score is calculated for each category: car, turtle, and horse.

[0150] Next, the recognition application 603 extracts the category with the highest score, and outputs the extraction result to the user application 605 via the application framework 604. Then, the user application 605 superimposes information indicating that the object is a car on the captured image, and displays the resulting captured image on a display or the like.

[0151] Fig.15 A second specific embodiment described in is human pose estimation in an image. Fig.15 In the embodiment illustrated in , two people appear in the captured image. First, the pre-trained AI model 601 performs inference on the image captured by the camera and outputs the inference result to the signal processing application 602 before formatting.

[0152] Next, the signal processing application 602 generates, for example, a heat map in the captured image as a formatted inference result, and outputs the heat map to the recognition application 603. At this point, key point information is still not available.

[0153] Next, the recognition application 603 performs post-processing of outputting the decoding and key point information to a formatted inference result, and outputs the posture information of the object as key point information to the user application 605 via the application framework 604. The user application 605 then superimposes information indicating the posture information on the captured image, and displays the resulting captured image on a display or the like.

[0154] As described above, the processing according to the present technology is performed. According to the present technology, without increasing the burden on the application developer, conversion to a format suitable for the target device 400t to which the application is deployed can be performed. This allows the application developer to efficiently develop an application using a system such as a market and provide the application to the user without being interfered by differences (hardware, OS, etc.) in the target device 400t.

[0155] Furthermore, in the present technology, the application can be converted into a format suitable for the target device 400t. Therefore, the application user can receive the application in an optimal format to run on the target device 400t.

[0156] Furthermore, by converting the application into AoT, processing time on the target device 400 t may be reduced and memory usage may be improved, and the application may be run on the target device 400 t even with a simple configuration or limited processing power.

[0157] In addition, by adding a signature to the converted application, the integrity of the application can be ensured.

[0158] [1-5. Description of GUI for device registration and application deployment]

[0159] Next, we will refer to Fig.16 and Fig.17 The registration GUI and the registration process for registering the device 400 in the application providing device 100 are described. The registration GUI is displayed on the user-side terminal device 300. Alternatively, the registration GUI may be displayed on the device 400. Fig.16 and Fig.17 A description is given assuming that the device 400 is a camera.

[0160] First, when the user selects and presses Fig.16 When the camera button 701 is registered in the menu list displayed on the left side of the registration GUI shown in A, as shown in FIG. Fig.16 As shown in (1) of A, a list of cameras registered in the application providing apparatus 100 is displayed on the registration GUI. A user who wants to register a new camera presses an add registration button 702.

[0161] Then, on the registration GUI, the Fig.16B shows a dialog box for a new camera. The user enters the name and certificate of the camera to be newly registered in this dialog box. The certificate is entered in the form of file selection, such as a pem file or a cer file. Note that in addition to or instead of the camera name and certificate, the user may be asked to enter other information.

[0162] When the name and certificate of the newly registered camera are input and the create button 703 is pressed, the name and certificate of the camera are transmitted from the user-side terminal device 300 to the application providing device 100 via the network. Then, when the name and certificate of the camera are stored in the device information database 105, the camera is registered as the device 400. Then, the dialog box is closed, and the registration GUI transitions to Fig.17 Status shown.

[0163] Then, if Fig.17 As shown in (2) in the figure, the new camera whose name and certificate have been entered is added to the list of registered devices on the registration GUI and displayed. This allows the user to confirm that his / her new camera has been registered.

[0164] Next, we will see Figures 18 to 20 A deployment GUI and a deployment process for deploying an application to a target device 400t are described. The deployment GUI is displayed on the user-side terminal device 300. Alternatively, the deployment GUI may be displayed on the target device 400t. In the drawings, it is assumed that the device 400 is a camera.

[0165] First, when the user selects and presses the application tab 801 on the deployment GUI, Fig.18 As shown in (3) in FIG. 1 , a list of applications that can be deployed to the camera is displayed. Such applications correspond to applications stored in the application database 102. Here, the user selects the application to be deployed and presses the deployment button 802. Fig.18 As shown, the selected application may be emphasized, for example, by utilizing highlighting.

[0166] Note that for application selection, the format of the application (WASM bytecode, AoT, Native) can be selected. In addition, only applications in the pre-selected format can be displayed in the application list. In addition, the user's desired conditions, such as whether to prioritize processing speed or whether security measures are required, can be selected by input, and the deployable application candidates obtained by filtering under the selected conditions can be displayed.

[0167] When the user selects an application to be deployed and presses the deploy button 802, a list of multiple cameras of the device 400 that are deployment targets is displayed next, such as Fig.19 As shown in (4) in Fig.19, the identification name and type of each camera are displayed. Here, the user selects a camera from the camera list as the target device 400t for deployment of the application. For example, the selection can be made by checking the corresponding check box. In the case where the camera selected by the user has already deployed the application, replacement is performed.

[0168] Note that multiple cameras can be grouped, and only cameras belonging to a specific group specified by the user can be displayed. Fig.19 In the example, only one group "Default" is displayed, and three cameras belonging to the group are displayed. The application can be deployed to any one of the multiple cameras being displayed or to all cameras in the multiple cameras belonging to one group at a time.

[0169] Note that in the list of camera candidates targeted for application deployment, filtering can be performed according to the format of the selected application, type of bytecode (whether the application can run on the camera), usage, code size, etc., and only appropriate cameras can be displayed.

[0170] Additionally, deployment parameters can be entered in json format etc if desired. This allows for additional settings for the deployment.

[0171] After selecting a camera for deployment of an application, when the user presses the deploy button 803, target device selection information is transmitted from the user-side terminal device 300 to the application providing device 100. Then, the providing-side deployment management unit 108 starts deploying the application to the camera as the target device 400t.

[0172] After starting the deployment, Fig. 20 As shown in (5) of FIG. 4 , the deployment status is displayed in the status section for the camera as the target device 400t. This allows the user to confirm the deployment status. Examples of the status include success, failure, cancellation, in progress, etc. Fig. 20 As shown in (6) in B, the deployment status can be displayed in the GUI displayed in response to selection of the manage camera menu.

[0173] If the application is not deployed, Fig.21 As shown, the user only needs to select and press the Applications tab 801 , select the application to be undeployed from the displayed application list, and press the Undeploy button 804 .

[0174] It should be noted that Figures 16 to 21 The names and layout of the buttons shown in the figure and the composition and layout of each display area on the GUI are merely examples, and the present technology is not limited to Figures 16 to 21 The content shown in .

[0175] <2. Modifications>

[0176] Although the details of the embodiments of the present technology have been described above, the present technology is not limited to the above-described embodiments, and various modifications based on the technical concept of the present technology are possible.

[0177] In the present embodiment, applications are provided, but are not limited to these applications, and AI models may also be provided. The present technology may be applied to programs, software, or application files that involve format conversion and are provided to the target device 400t.

[0178] The application database 102 and the device information database 105 may be configured in an external storage device instead of being included in the application providing device 100. In this case, the application selecting unit 104 and the device information selecting unit 106 access the external storage device to acquire the application and the target device information.

[0179] In the present embodiment, a case has been described where the user selects the target device 400t for deployment of the application, but the target device 400t to which the application providing apparatus 100 deploys the application may be automatically selected instead of the user's selection. For example, an AI-based determination is made based on accumulated information about past relationships between applications and devices 400; specifically, a specific type of application is deployed to a specific type of device 400.

[0180] The present technology may also have the following configurations. (1)

[0182] An application providing device, comprising:

[0183] The application conversion unit converts the application retrieved from the application database according to the device information retrieved from the device information database as information about a device that executes the application on the device. (2)

[0185] The application providing apparatus according to (1), further including: an application selecting unit that retrieves the application from the application database in response to a request from a user of the application, and outputs the application to the application converting unit. (3)

[0187] The application providing device according to (1) or (2), further includes: a device information selection unit that retrieves device information about a device running the application on a device selected by a user of the application from a device information database and outputs the device information to the application conversion unit. (4)

[0189] The application providing apparatus according to any one of (1) to (3), wherein the application conversion unit switches a format of the application subjected to conversion. (5)

[0191] The application providing device according to (4), wherein the application conversion unit switches the format of the application based on whether a processing speed of the application is prioritized on the device. (6)

[0193] The application providing device according to (4) or (5), wherein the application conversion unit switches the format of the application based on whether the application on the device requires security measures. (7)

[0195] The application providing device according to any one of (4) to (6), wherein the application conversion unit switches the format of the application based on a configuration of a device that executes the application. (8)

[0197] The application providing apparatus according to any one of (1) to (7), wherein the application is in an AoT format. (9)

[0199] The application providing device according to any one of (1) to (8), wherein the application is in a native format. (10)

[0201] The application providing device according to any one of (1) to (9), wherein the application is in WASM format. (11)

[0203] The application providing device according to any one of (1) to (10) further includes: a deployment management unit that deploys the application converted by the application conversion unit to the device. (12)

[0205] The application providing device according to any one of (1) to (11), further comprising: a user and device management unit that stores device information input by a user in a device information database. (13)

[0207] The application providing apparatus according to any one of (1) to (12), wherein the application conversion unit adds the signature to the converted application. (14)

[0209] The application providing device according to (13), wherein the application for which the signature verification fails is not installed on the device. (15)

[0211] The application providing device according to (13), wherein an application for which the signature verification fails is not executed on the device. (16)

[0213] The application providing device according to any one of (1) to (15), wherein the application converted by the application conversion unit has a code for checking a memory of the device to be accessed. (17)

[0215] The application providing device according to any one of (1) to (16), wherein a GUI for registering the device in the device information database is displayed on a terminal device used by a user of the application or the device. (18)

[0217] The application providing device according to any one of (1) to (17), wherein a GUI for deploying the application to the device is displayed on a terminal device used by a user of the application program or the device. (19)

[0219] An application providing method, comprising:

[0220] The application retrieved from the application database is converted according to the device information retrieved from the device information database as information about a device running the application on the device. (20)

[0222] A program for causing a computer to execute an application providing method, the method comprising:

[0223] The application retrieved from the application database is converted according to the device information retrieved from the device information database as information about a device running the application on the device.

[0224] Reference Symbols List

[0225] 100 Application providing device

[0226] 102 Application Database

[0227] 103 User and Device Management Unit

[0228] 104 Application Selection Unit

[0229] 105 Device Information Database

[0230] 106 Device information selection unit

[0231] 107 Application Conversion Unit

[0232] 108 provides a side deployment management unit.

Claims

1. An application providing device, comprising: The application conversion unit converts the application retrieved from the application database according to the device information retrieved from the device information database as information about the device on which the application is executed.

2. The application providing device according to claim 1, further comprising: An application selection unit retrieves the application from the application database in response to a request from a user of the application, and outputs the application to the application conversion unit.

3. The application providing device according to claim 1, further comprising: A device information selection unit retrieves device information of a device on which an application is executed on a device selected by a user of the application from a device information database, and outputs the device information to the application conversion unit.

4. The application providing device according to claim 1, wherein: The application conversion unit switches the format of the application subjected to conversion.

5. The application providing device according to claim 4, wherein: The application conversion unit switches the format of the application based on whether a processing speed of the application is prioritized on the device.

6. The application providing device according to claim 4, wherein: The application conversion unit switches a format of the application based on whether the application on the device requires security measures.

7. The application providing device according to claim 4, wherein: The application conversion unit switches a format of the application based on a configuration of the device that executes the application.

8. The application providing device according to claim 1, wherein: The application is in AoT format.

9. The application providing device according to claim 1, wherein: The application is in Native format.

10. The application providing device according to claim 1, wherein: The application is in WASM format.

11. The application providing device according to claim 1, further comprising: A deployment management unit deploys the application converted by the application conversion unit to the device.

12. The application providing device according to claim 1, further comprising: The user and device management unit stores the device information input by the user in the device information database.

13. The application providing device according to claim 1, wherein: The application conversion unit adds a signature to the application subjected to conversion.

14. The application providing device according to claim 13, wherein: Applications that fail signature verification are not installed on the device.

15. The application providing device according to claim 13, wherein: Applications that fail signature verification do not run on the device.

16. The application providing device according to claim 1, wherein: The application converted by the application conversion unit has a code for checking a memory of a device to be accessed.

17. The application providing device according to claim 1, wherein: A GUI for registering the device in the device information database is displayed on a terminal device used by a user of the application or the device.

18. The application providing device according to claim 1, wherein: A GUI for deploying the application to the device is displayed on a terminal device used by a user of the application or the device.

19. A method for providing an application, comprising: The application retrieved from the application database is converted according to the device information retrieved from the device information database as information about the device on which the application is executed.

20. A program for causing a computer to execute an application providing method, the application providing method comprising: The application retrieved from the application database is converted according to the device information retrieved from the device information database as information about the device on which the application is executed.

Images