Information processing method, server device and information processing device

Abstract

The present technology relates to a method for processing information, a server device, and an information processing device that allows only a legitimate user to use the device. In a method for processing information according to the present technology, a server device matches first device information unique to a device that has acquired connection destination information indicating a connection destination, and second device information registered in advance, to authenticate the device. The present technology can be applied, for example, to an information processing system that provides a cloud service using an AI application.

Description

Technical Field

[0001] The present technology relates to an information processing method, a server device, and an information processing device, and more particularly, to an information processing method, a server device, and an information processing device that allow only authorized users to use the device. Background Art

[0002] A two-dimensional code is generally used to set up a device. For example, Patent Document 1 discloses that by using a camera to image a sheet of paper on which the unique information of a user (ID and password) is recorded in the form of a two-dimensional code, and storing the unique information (ID and password) obtained by analyzing the two-dimensional code, the setting of connecting the device is performed.

[0003] [Citation List]

[0004] [Patent Document]

[0005] [Patent Document 1] JP 2012-155754 A Summary of the Invention

[0006] [Technical Problem]

[0007] Before the unique information is input, the device does not hold information on the user who restricts the use of the device. Therefore, a person different from the authorized user who purchased the device can use the device by inputting the unique information of that person.

[0008] The present technology has been devised in such a situation and allows only authorized users to use the device.

[0009] [Solution to the Problem]

[0010] An information processing method according to a first aspect of the present technology includes: performing authentication of a device by comparing, via a server device, device-unique first device information transmitted from a device that has acquired connection destination information indicating a connection destination with second device information registered in advance.

[0011] A server device according to a first aspect of the present technology includes an authentication unit that performs authentication of a device by comparing device-unique first device information transmitted from a device that has acquired connection destination information indicating a connection destination with second device information registered in advance.

[0012] The information processing method according to the second aspect of the present technology includes: obtaining connection destination information via an information processing device, the connection destination information indicating a server as the connection destination, the server performing authentication of its own device by comparing the unique first device information of its own device with the pre-registered second device information; transmitting the first device information to the server indicated by the connection destination information; and downloading the firmware of its own device from the server when the first device information matches the second device information.

[0013] The information processing device according to the second aspect of the present technology includes: an acquisition unit that acquires connection destination information, the connection destination information indicating a server as the connection destination, the server performing authentication of its own device by comparing the unique first device information of its own device with the pre-registered second device information; and a communication control unit that transmits the first device information to the server and downloads the firmware of its own device from the server when the first device information matches the second device information.

[0014] In the first aspect of the present technology, device authentication is performed by comparing the unique first device information transmitted from a device that has acquired connection destination information indicating the connection destination with the pre-registered second device information.

[0015] In the second aspect of the present technology, connection destination information is obtained, the connection destination information indicating a server as the connection destination, the server performing authentication of its own device by comparing the unique first device information of its own device with the pre-registered second device information; the first device information is transmitted to the server indicated by the connection destination information; and when the first device information matches the second device information, the firmware of its own device is downloaded from the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a diagram showing a schematic configuration example of an information processing system according to the present technology.

[0017] Figure 2 is a diagram for describing each device for registering and downloading an AI model and an AI application via a market function included in a cloud-side information processing device.

[0018] Figure 3 is a diagram showing when registering or downloading an AI model or an AI application via the market function with Figure 4 is a diagram showing an example of a process of processing performed by each device.

[0019] Figure 4 is a diagram showing when registering or downloading an AI model or an AI application via the market function with Figure 3Diagram showing an example of the process flow executed by each device when registering or downloading an AI model or AI application together.

[0020] Figure 5 Diagram for describing the connection mode between the cloud-side information processing device and the edge-side information processing device.

[0021] Figure 6 Showing an overview of the functional configurations of the cloud server and the management server.

[0022] Figure 7 Block diagram showing an example of the configuration of a camera.

[0023] Figure 8 Diagram showing an example of the configuration of an image sensor.

[0024] Figure 9 Block diagram showing the software configuration of a camera.

[0025] Figure 10 Block diagram showing the operating environment of a container in the case of using container technology.

[0026] Figure 11 Block diagram showing an example of the hardware configuration of an information processing device.

[0027] Figure 12 Diagram showing the configuration of each device related to the settings of a camera.

[0028] Figure 13 Diagram for describing the detailed process of camera settings.

[0029] Figure 14 Diagram showing an example of a QR code creation screen.

[0030] Figure 15 Diagram for describing the detailed process of camera settings.

[0031] Figure 16 Diagram showing an example of the process of camera authentication.

[0032] Figure 17 Sequence diagram for describing the process executed by the edge-side information processing device and the cloud-side information processing device for camera settings.

[0033] Figure 18 Diagram showing an example of another process of camera authentication.

[0034] Figure 19 Diagram for describing the process of relearning processing.

[0035] Figure 20It is a diagram showing an example of a login screen for logging in to the market.

[0036] Figure 21 It is a diagram showing an example of a developer-oriented screen presented to each developer using the market.

[0037] Figure 22 It is a diagram showing an example of a user-oriented screen presented to application-using users who use the market. Detailed Description of the Invention

[0038] Hereinafter, embodiments for implementing the present technology will be described. The description will be made in the following order.

[0039] 1. Overview of the information processing system

[0040] 1-1. Overall configuration of the information processing system

[0041] 1-2. Registration of the AI model and AI applications

[0042] 1-3. Overview of the functions of the information processing system

[0043] 1-4. Configuration of the camera

[0044] 1-5. Hardware configuration of the information processing device

[0045] 2. Regarding camera settings

[0046] 3. Regarding re-learning

[0047] 4. Screen examples of the market

[0048] <1. Overview of the information processing system>

[0049] <1-1. Overall configuration of the information processing system>

[0050] Figure 1 It is a block diagram showing a schematic configuration example of an information processing system 100 according to the present technology.

[0051] As Figure 1 shown, the information processing system 100 includes at least a cloud server 1, a user terminal 2, a plurality of cameras 3, and a management server 4. In this example, the cloud server 1, the user terminal 2, the camera 3, and the management server 4 are configured to be able to communicate with each other via a network 5 such as the Internet.

[0052] The cloud server 1, the user terminal 2, and the management server 4 are configured as information processing devices including a microcomputer that includes a central processing unit (CPU), a read-only memory (ROM), and a random access memory (RAM).

[0053] Here, the user terminal 2 is an information processing device assumed to be used by a user who is a recipient of the services of the information processing system 100. In addition, the management server 4 is an information processing device assumed to be used by a service provider.

[0054] Each camera 3 includes, for example, an image sensor such as a charge-coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor, and captures a subject to obtain image data (captured image data) as digital data.

[0055] In addition, as will be described later, each camera 3 also has a function of processing a captured image using artificial intelligence (AI) (such as image recognition processing, image detection processing, etc.). In the following description, various types of processing of images (such as image recognition processing and image detection processing) will be simply referred to as "image processing". For example, various types of processing of an image using AI (or an AI model) will be described as "AI image processing".

[0056] Each camera 3 can transmit various data such as processing result information indicating the result of processing (such as image processing) using AI to the cloud server 1 or the management server 4, and can receive various data transmitted from the cloud server 1 or the management server 4.

[0057] Here, for example, it is assumed that the information processing system 100 shown in Figure 1 is used such that the cloud server 1 generates analysis information of an object based on the processing result information obtained by the image processing of each camera 3 and allows the user to browse the generated analysis information via the user terminal 2.

[0058] In this case, as applications of each camera 3, various applications of surveillance cameras can be considered. Examples of cameras include surveillance cameras for indoor use (such as in stores, offices, and houses), surveillance cameras for outdoor use (such as in parking lots and on streets) (including traffic surveillance cameras, etc.), surveillance cameras for production lines in factory automation (FA) and industrial automation (IA), surveillance cameras for monitoring the inside and outside of vehicles, etc.

[0059] For example, in the case of applying as a surveillance camera in a store, it is conceivable to arrange a plurality of cameras 3 at predetermined positions in the store so that a user can check the customer group of customers (gender, age group, etc.), the actions in the store (flow line), etc. In this case, as the above analysis information, it is conceivable to generate information on the customer group of customers, information on the flow line in the store, information on the congestion state at the checkout registration desk (for example, the waiting time at the checkout registration desk), etc. Alternatively, in the case of applying as a traffic surveillance camera, it is considered to arrange each camera 3 at each position near the road, and the user can identify information such as the number of overtaking vehicles (vehicle numbers), the color of the car, the vehicle type, etc. In this case, it is conceivable to generate information such as the number, color, and vehicle type of cars as the above analysis information.

[0060] In addition, in the case of using a traffic surveillance camera in a parking lot, it is conceivable to arrange the camera so that it can monitor each parked vehicle, monitor whether a suspicious person performing a suspicious act is around each vehicle, and in the case of the presence of a suspicious person, notify the presence of the suspicious person, the attributes of the suspicious person (gender or age group), etc. In addition, it is also conceivable to monitor the empty spaces in the town or the parking lot and notify the user of the location of the spaces where vehicles can be parked.

[0061] In the following description, various devices can be roughly divided into a cloud-side information processing device and an edge-side information processing device. The cloud server 1 and the management server 4 correspond to the cloud-side information processing device, and the cloud-side information processing device is a group of devices that provide services assumed to be used by multiple users.

[0062] In addition, the camera 3 corresponds to the edge-side information processing device, and the edge-side information processing device can be regarded as a group of devices arranged in the environment prepared by the user using the cloud service.

[0063] However, both the cloud-side information processing device and the edge-side information processing device can be in the environment prepared by the same user.

[0064] Note that each camera 3 can be connected to the network 5 via a fog server. In the case of setting up a fog server, each camera 3 is configured to be able to perform data communication with the fog server and be able to transmit various types of data to the fog server or receive various types of data transmitted from the fog server.

[0065] The fog light server corresponds to the edge-side information processing device and is assumed to be used to generate analysis information of an object, rather than the cloud server 1.

[0066] Assume that the fog server is arranged for each monitoring target. For example, in the above application of monitoring a store, it is arranged in the store together with each camera 3 as a monitoring target. In this way, by setting up fog servers for each monitoring object such as a store, the cloud server 1 does not need to directly receive the transmission data from the multiple cameras 3 of the monitoring object, reducing the processing burden on the cloud server 1.

[0067] Note that in the case where there are multiple stores as monitoring targets and all stores belong to the same series, one fog server can be provided for multiple stores instead of one fog server for each store. That is, one fog server is not limited to being provided for each monitoring target, and one fog server can be provided for multiple monitoring targets. Note that the fog server can be an in-premises server.

[0068] <1-2. Registration of AI Model and AI Application>

[0069] As described above, in the information processing system 100, in the camera 3 as an edge-side information processing device, AI image processing is performed, and in the cloud server 1 as a cloud-side information processing device, the result information of the AI image processing on the edge side (for example, the result information of the image recognition processing using AI) is used to implement advanced application functions.

[0070] Here, various methods for registering application functions in the cloud server 1 as a cloud-side information processing device can be considered. Examples will be described in detail with reference to Figure 2 Examples.

[0071] The above cloud server 1 and management server 4 are information processing devices that constitute the cloud-side environment. In addition, the camera 3 is an information processing device that constitutes the environment on the edge side.

[0072] Note that the camera 3 can be regarded as a device including a control unit that executes overall control of the camera 3, and the camera 3 can be regarded as a device including another device, which is an image sensor IS. The image sensor IS includes an arithmetic processing unit that performs various types of processing (including AI image processing) on the captured image. That is, it can be considered that the image sensor IS (i.e., another edge-side information processing device) is installed inside the camera 3 (i.e., the edge-side information processing device).

[0073] In addition, examples of the user terminal 2 used by users who use various services provided by the cloud-side information processing device include an application developer terminal 2A used by users who develop applications for AI image processing, an application user terminal 2B used by users who use the application, and an AI model developer terminal 2C used by users who develop AI models for AI image processing. Note that, of course, the application developer terminal 2A can be used by users who develop applications that do not use AI image processing.

[0074] In the cloud-side information processing device, a learning data set for performing learning by AI is prepared. A user who develops an AI model communicates with the cloud-side information processing device using the AI model developer terminal 2C and downloads the learning data set. At this time, the learning data set can be provided for a fee. For example, an AI model developer can purchase the learning data set in a state where different functions and materials registered in a market (electronic market) can be purchased, and the market is prepared as a cloud-side function by registering personal information in the market.

[0075] After developing an AI model using the learning data set, the AI model developer registers the developed AI model in the market using the AI model developer terminal 2C. Therefore, when the AI model is downloaded, an incentive can be paid to the AI model developer.

[0076] In addition, a user who develops an application (application development user) downloads the AI model from the market using the application developer terminal 2A and develops an application using the AI model (hereinafter referred to as an "AI application"). At this time, as described above, an incentive can be paid to the AI model developer.

[0077] The application development user registers the developed AI application in the market using the application developer terminal 2A. Therefore, when the application is downloaded, an incentive can be paid to the user who has developed the AI application.

[0078] A user who uses the AI application (application user) performs an operation of deploying the AI application and the AI model from the market to the camera 3, which is an edge-side information processing device managed as a user, using the application user terminal 2B. At this time, an incentive can be paid to the AI model developer. When the AI application and the AI model are deployed in the camera 3, AI image processing using the AI application and the AI model can be performed in the camera 3, and not only can an image be captured, but also customer detection and vehicle detection can be performed through AI image processing.

[0079] Here, the deployment of the AI application and the AI model indicates that the AI application and the AI model are installed on the target as execution subjects, so that the target (device) as the execution subject can use the AI application and the AI model. In other words, at least a part of the program as the AI application can be executed.

[0080] In addition, in the camera 3, attribute information of a customer can be extracted from a captured image captured by the camera 3 through AI image processing. This attribute information is transmitted from the camera 3 to the cloud-side information processing device via the network 5.

[0081] A cloud application is deployed in the cloud - side information processing device, and each user can use this cloud application through the network 5. Then, in the cloud application, an application for analyzing the customer's traffic line by using attribute information or the captured image of the customer is prepared. Such a cloud application is uploaded by application - developing users, etc.

[0082] By using the application user terminal 2B to use the cloud application for traffic - line analysis, the application - using user can perform traffic - line analysis of customers for his own store and browse the analysis results. The browsing of the analysis results is performed, for example, by graphically presenting the customer's traffic line on the map of the store. In addition, the browsing of the analysis results can be performed by displaying the results of the traffic - line analysis in the form of a heat map and presenting the density of customers, etc. In addition, this information can be classified and displayed according to the customer's attribute information.

[0083] In the cloud - side market, an AI model optimized for each user can be registered. For example, the captured image captured by the camera 3 arranged in the store managed by a specific user is appropriately uploaded and accumulated in the cloud - side information processing device.

[0084] In the cloud - side information processing device, each time a certain number of uploaded captured images are accumulated, a re - learning process of the AI model is performed, and a process of updating the AI model in the market and re - registering the AI model is performed. Note that the re - learning process of the AI model can be selected by the user as an option in the market.

[0085] For example, by configuring an AI model in the camera 3 that is re - trained using the dark images from the camera 3 arranged in the store, the recognition rate of image processing for the captured images taken in the dark can be improved. In addition, by configuring an AI model in the camera 3 that is re - trained using the bright images from the camera 3 arranged outside the store, the recognition rate of image processing for the captured images in the bright place can be improved. That is, the application - using user can always obtain optimized processing result information by redeploying the updated AI model in the camera 3 again. In addition, the re - learning process of the AI model will be described later.

[0086] In addition, in the case where personal information is included in the information (captured images, etc.) uploaded from the camera 3 to the cloud - side information processing device, data with privacy - related information deleted can be uploaded from the perspective of privacy protection, or data with privacy - related information deleted can be made available to AI - model - developing users or application - developing users.

[0087] In Figure 3 and Figure 4 The flowcharts show the flow of the above - mentioned processing. It should be noted that the cloud - side information processing device corresponds to Figure 1 the cloud server 1, the management server 4, etc. in

[0088] In response to the AI model developer browsing the list of datasets registered in the market and selecting a desired dataset by using the AI model developer terminal 2C having a display unit including a liquid crystal display (LCD), an organic electroluminescence (EL) panel, etc., in step S21, the AI model developer terminal 2C transmits a download request for the selected dataset to the cloud-side information processing device ( Figure 3 ).

[0089] In response to this, in step S1, the cloud-side information processing device accepts the request, and in step S2, performs a process of transmitting the requested dataset to the AI model developer terminal 2C.

[0090] In step S22, the AI model developer terminal 2C performs a process of receiving the dataset. Thus, the AI model developer can use the dataset to develop an AI model.

[0091] After the AI model developer completes the development of the AI model, when the AI model developer performs an operation for registering the developed AI model in the market (for example, specifying the name of the AI model, the address where the AI model is placed, etc.), in step S23, the AI model developer terminal 2C transmits a registration request for the AI model in the market to the cloud-side information processing device.

[0092] In response to this, the cloud-side information processing device can display the AI model in the market, for example, by accepting the registration request in step S3 and performing the registration process of the AI model in step S4. Thus, users other than the AI model developer can download the AI model from the market.

[0093] For example, an application developer who wants to develop an AI application browses the list of AI models registered in the market by using the application developer terminal 2A. In response to the operation of the application developer (for example, an operation of selecting one of the AI models in the market), in step S31, the application developer terminal 2A transmits a download request for the selected AI model to the cloud-side information processing device.

[0094] The cloud-side information processing device accepts the request in step S5 and transmits the AI model to the application developer terminal 2A in step S6.

[0095] In step S32, the application developer terminal 2A receives the AI model. As a result, the application developer can use the AI model developed by others to develop an AI application.

[0096] After the application developer completes the development of the AI application, when the application developer performs an operation for registering the AI application in the market (for example, an operation of specifying the name of the AI application, the address where the AI model is placed, etc.), the application developer terminal 2A transmits a registration request for the AI application to the cloud-side information processing device in step S33.

[0097] For example, by accepting the registration request in step S7 and registering the AI application in step S8, the cloud-side information processing device can display the AI application in the market. Therefore, users other than the application developer can select and download the AI application in the market.

[0098] Next, in the camera 3, settings are made in step S51 ( Figure 4 ). In the settings of the camera 3, a two-dimensional code (for example, a QR code (registered trademark)) is captured using the image sensor IS, and the two-dimensional code appearing in the captured image is analyzed to obtain connection destination information indicating the connection destination of the camera 3. Here, for example, the management server 4 is the connection destination. In the memory of the management server 4, for example, firmware (FW) for the camera 3 to execute the AI application and the like is stored. The camera 3 acquires the FW from the memory of the management server 4 and updates the FW to the FW, thereby completing the settings of the camera 3.

[0099] In addition, the settings of the camera 3 will be described again later.

[0100] After the settings of the camera 3 are completed, the application user terminal 2B performs purpose selection in step S41 according to the operation of the user who intends to use the AI application. In the purpose selection, the selected purpose is transmitted to the cloud-side information processing device.

[0101] In response to this, the cloud-side information processing device selects the AI application according to the purpose in step S9, and performs a preparation process (deployment preparation process) for deploying the AI application or the AI model to each device in step S10. In the deployment preparation process, it is determined which device will execute each software (SW) component that constitutes the AI application for realizing the function desired by the user.

[0102] Each SW component can be a container to be described later, or can be a microservice. Note that the SW component can also be implemented by using a web assembly technology.

[0103] In the case of an AI application that counts the number of customers for each attribute such as gender and age, it includes an SW component that uses an AI model to detect a person's face from the captured image, an SW component that extracts the person's attribute information from the detection result of the face, an SW component that aggregates the results, an SW component that visualizes the aggregated results, and the like.

[0104] In step S11, the cloud - side information processing device performs the process of deploying each SW component in each device. In this process, the AI application and the AI model are transmitted to each device such as camera 3.

[0105] In response to this, in camera 3, the deployment process of the AI application and the AI model is performed in step S52. Thus, AI image processing can be performed on the captured image captured by camera 3. Note that in the case where a fog server is set up, similarly in the fog server, the deployment process of the AI application and the AI model is performed as needed.

[0106] However, in the case where all types of processing are performed in camera 3, the deployment process of the fog server is not performed.

[0107] In step S53, camera 3 acquires an image (captured image) by performing an imaging operation. Then, in step S54, camera 3 performs AI image processing on the acquired captured image and obtains, for example, an image recognition result.

[0108] In step S55, camera 3 performs the transmission process of the captured image and the result information of the AI image processing. In the information transmission in step S55, both the captured image and the result information of the AI image processing can be transmitted, or only one of the captured image and the result information of the AI image processing can be transmitted.

[0109] The cloud - side information processing device that receives this information performs an analysis process in step S12. Through this analysis process, for example, traffic line analysis of customers, vehicle analysis processing for traffic monitoring, etc. are performed.

[0110] In step S13, the cloud - side information processing device performs the presentation process of the analysis result. This process is implemented, for example, by the user using the above - mentioned cloud application.

[0111] When receiving the presentation process of the analysis result, the application user terminal 2B performs the process of displaying the analysis result on a monitor or the like in step S42.

[0112] Through the processing so far, the user of the AI application can obtain the analysis result according to the purpose selected in step S41.

[0113] It should be noted that in the cloud - side information processing device, the AI model can be updated after step S13. By updating and deploying the AI model, an analysis result suitable for the user's usage environment can be obtained.

[0114] <1 - 3. Summary of the functions of the information processing system>

[0115] In this embodiment, as a service using the information processing system 100, it is assumed that a user who is a customer can select a service for the type of AI image processing function for each camera 3. For the selection of the function type, for example, an image recognition function, an image detection function, etc. can be selected, or a more detailed type can be selected to present an image recognition function or an image detection function for a specific object.

[0116] For example, as a business model, a service provider sells a camera 3 or a fog server with an image recognition function to a user through AI, and installs the camera 3 or the fog server at a location to be monitored. Then, a service for providing the above analysis information to the user is deployed.

[0117] At this time, since the applications required by the system are different for each customer, such as applications for store monitoring and traffic monitoring, the AI image processing function of the camera 3 can be selectively set so that analysis information corresponding to the applications required by the customer can be obtained. In addition, when the authentication of the camera 3 is successful so that only authorized users who purchase the camera 3 can use the camera 3, the setting of the camera 3 can be performed.

[0118] In this example, the management server 4 has a function of authenticating the camera 3. In addition, the cloud server 1 or the fog server can also have the function of the management server 4.

[0119] Here, reference will be made to Figure 5 the connection between the cloud server 1 and the management server 4 as cloud-side information processing devices and the camera 3 as an edge-side information processing device.

[0120] In the cloud-side information processing device, a re-learning function, a device management function, and a market function that are functions available via the Hub are implemented.

[0121] The Hub performs highly reliable communication protected by security with respect to the edge-side information processing device. As a result, various functions can be provided to the edge-side information processing device.

[0122] The re-learning function is a function that performs re-learning and provides a newly optimized AI model, thereby providing an appropriate AI model based on new learning materials.

[0123] The device management function is a function of managing the camera 3, etc. as edge-side information processing devices, and can provide functions such as managing and monitoring the AI models deployed in the camera 3, and detecting and troubleshooting problems, for example.

[0124] In addition, the device management function is also a function for managing the information of the camera 3 and the fog light server. The information of the camera 3 and the fog light server includes information of the chip used as the arithmetic processing unit, information such as the storage capacity and usage rate of the CPU and memory, and information of the software such as the operating system (OS) installed in each device.

[0125] In addition, the device management function protects the secure access of authenticated users.

[0126] The market function provides functions for registering the AI models developed by the above-mentioned AI model developers and the AI applications developed by the application developers, functions for deploying these developments to the permitted edge-side information processing devices, etc. In addition, the market function also provides functions related to the payment of rewards according to the developed deployment.

[0127] The camera 3 as an edge-side information processing device includes an edge runtime, an AI application, an AI model, and an image sensor 1S.

[0128] The edge runtime serves as embedded software for managing the applications deployed in the camera 3 and communicating with the cloud-side information processing device.

[0129] As described above, the AI models and AI applications are obtained by deploying the AI models and AI applications registered in the market in the cloud-side information processing device, and thus, the camera 3 can obtain the result information of AI image processing according to the purpose using the captured images.

[0130] Reference will be made to Figure 6 describe the outline of the functional configurations of the cloud server 1 and the management server 4.

[0131] As Figure 6 shown in A of, the cloud server 1 includes an account service unit 11, a device monitoring unit 12, a market unit 13, and a camera service unit 14.

[0132] The account service unit 11 has a function of generating and managing the account information of users. The account service unit 11 accepts the input of user information and generates account information based on the input user information (generates account information including at least user ID and password information).

[0133] In addition, the account service unit 11 accepts the input of network information, etc. of the camera 3 for connecting to the network 5, and generates a QR code for setting the camera 3 based on the input information, etc.

[0134] In addition, the account service unit 11 also performs registration processing (registration of account information) on the AI model developers and the AI application developers (hereinafter, also simply referred to as "software developers").

[0135] The device monitoring unit 12 has a function of executing a process for monitoring the usage status of the camera 3. For example, as various elements related to the usage status of the camera 3, such as the usage location of the camera 3, the output frequency of the output data of the AI image processing, the free space of the CPU and memory for the AI image processing, etc., information such as the usage rates of the above CPU and memory is monitored.

[0136] The market unit 13 has a function of selling AI models and AI applications. For example, users can purchase AI applications and the AI models used by the AI applications via the sales WEB site (sales site) provided by the market unit 13. In addition, software developers can purchase the AI models for creating AI applications via the above sales site.

[0137] The camera service unit 14 has a function of providing services related to the usage of the camera 3 to users. As one of the functions of the camera service unit 14, for example, a function related to the generation of the above analysis information can be exemplified. That is, the camera service department 14 generates analysis information of an object based on the processing result information of the image processing of the camera 3, and performs a process of allowing the user to view the generated analysis information via the user terminal 2.

[0138] In addition, the function of the camera service unit 14 includes an imaging setting search function. Specifically, the imaging setting search function is a function of obtaining the recognition result information of the AI image processing from the camera 3, and using AI to search for the imaging setting information of the camera 3 based on the obtained recognition result information. Here, the imaging setting information generally refers to the setting information related to the imaging operation for obtaining a captured image. Specifically, it widely includes optical settings such as focus and aperture, settings related to the readout operation of the captured image signal such as frame rate, exposure time, and gain, and settings related to the image signal processing of the read captured image signal such as gamma correction processing, noise reduction processing, and super-resolution processing.

[0139] In addition, the function of the camera service unit 14 also includes an AI model search function. The AI model search function is a function of obtaining the recognition result information of the AI image processing from the camera 3, and using AI to search for the optimal AI model to be used for the AI image processing in the camera 3 based on the obtained recognition result information. Here, searching for an AI model means, for example, in the case where the AI image processing is implemented by a convolutional neural network (CNN) including a convolutional operation, processing for optimizing various processing parameters such as weighting factors, setting information related to the neural network structure (for example, information including the kernel size), etc.

[0140] In addition, the functions of the camera service unit 14 also include a processing sharing determination function. In the processing sharing determination function, when an AI application is deployed in the edge-side information processing device, the process of determining the device as the deployment destination in units of SW components is performed as the above-mentioned deployment preparation process. Note that some SW components can be determined to be executed in the device on the cloud side, and in this case, assuming that some SW components have been deployed in the device on the cloud side, the deployment process may not be executed.

[0141] For example, as in the above example, in the case of an AI application including an SW component for detecting a person's face, an SW component for extracting the person's attribute information, an SW component for aggregating the extraction results, and an SW component for visualizing the aggregation results, the camera service unit 14 determines the image sensor IS of the camera 3 as the device serving as the deployment destination. For the SW component for detecting a person's face, the camera 3 is determined as the device serving as the deployment destination. For the SW component for extracting the person's attribute information, for the SW component for aggregating the extraction results, the fog server is determined as the device serving as the deployment destination, and it is determined that the SW component for visualizing the aggregation results is executed in the cloud server 1 without newly deploying the SW component to the device.

[0142] By having the imaging setting search function and the AI model search function as described above, it is possible to perform an imaging setting that makes the result of AI image processing good, and it is possible to perform AI image processing using an appropriate AI model according to the actual usage environment. In addition, by having the processing sharing determination function, AI image processing and its analysis processing can be executed in an appropriate device.

[0143] Note that the camera service unit 14 has an application setting function. The application setting function is a function of setting an appropriate AI application according to the user's purpose. For example, the camera service unit 14 selects an appropriate AI application in response to the user's selection of an application such as store monitoring or traffic monitoring. As a result, the SW components constituting the AI application are naturally also determined.

[0144] In the application setting function, the process of accepting the operation of the user selecting the purpose (application) in the user terminal 2 (corresponding to Figure 2 the application user terminal 2B in

[0145] Next, as shown in Figure 6 B of

[0146] The license authorization unit 21 has the function of performing processes related to various authentications. Specifically, the license authorization unit 21 executes processes related to the authentication of each camera 3 and processes related to the authentication of each of the AI model, SW, and FW used by the camera 3.

[0147] Here, the above-mentioned SW refers to the SW that appropriately implements the AI image processing in the camera 3. In order to appropriately execute AI image processing based on the captured image and transmit the result of the AI image processing to the fog lamp server or the cloud server 1 in an appropriate format, it is necessary to control the data input to the AI model and appropriately process the output data of the AI model. The above-mentioned SW is the SW that includes the peripheral processes required to appropriately implement the AI image processing. This SW is the SW for using the AI model to implement the desired function and corresponds to the above-mentioned AI application.

[0148] Note that the AI application is not limited to an application that uses only one AI model, and an application that uses two or more AI models can also be considered. For example, there may be an AI application with a processing flow in which information on the recognition result (image data, etc., hereinafter referred to as "recognition result information") obtained by an AI model that performs AI image processing by using a captured image as input data is input to another AI model and the second AI image processing is performed.

[0149] For the authentication of the camera 3, in the license authorization unit 21, when the license authorization unit 21 and the camera 3 are connected via the network 6, a process of comparing the device information unique to the camera 3 transmitted from the camera 3 with the pre-registered device information is performed. The device information of the camera 3 purchased by the user is registered in the license authorization unit 21, for example, in association with the user.

[0150] In addition, regarding the authentication of the AI model and SW, for each of the AI applications and AI models registered from the application developer terminal 2A and the AI model developer terminal 2C, a process of issuing unique IDs (AI model ID, software ID) is performed.

[0151] Furthermore, in the license authorization unit 21, the following processes are performed: various keys, certificates, etc. for enabling secure communication between the camera 3, the application developer terminal 2A, the AI model developer terminal 2C, etc. and the cloud server 1 are issued to the manufacturer of the camera 3 (specifically, the manufacturer of the image sensor IS to be described later), the AI application developer, the AI model developer, etc., and a process for updating or stopping the validity of the certificate is also performed.

[0152] The memory 22 stores the FW downloaded by the camera 3 at the time of setting.

[0153] <1-4. Configuration of the Camera>

[0154] Figure 7 is a block diagram showing an example configuration of the camera 3.

[0155] As Figure 7 shown, the camera 3 includes an imaging optical system 31, an optical system driving unit 32, an image sensor 1S, a control unit 33, a memory unit 34, and a communication unit 35. The image sensor IS, the control unit 33, the memory unit 34, and the communication unit 35 are connected via a bus 36 and can perform data communication with each other.

[0156] The imaging optical system 31 includes lenses such as a cover lens, a zoom lens, and a focusing lens, and a diaphragm (iris) mechanism. Light (incident light) from an object is guided by the imaging optical system 31 and converged on the light receiving surface of the image sensor IS.

[0157] The optical system driving unit 32 comprehensively instructs driving units of the zoom lens, the focusing lens, and the diaphragm mechanism included in the imaging optical system 31. Specifically, the optical system driving unit 32 includes actuators for driving each of the zoom lens, the focusing lens, and the diaphragm mechanism, and a driving circuit for the actuators.

[0158] The control unit 33 includes, for example, a microcomputer including a CPU, a ROM, and a RAM, and performs overall control of the camera 3 by the CPU executing various types of processing according to a program stored in the ROM or a program loaded into the RAM.

[0159] In addition, the control unit 33 gives driving instructions for the zoom lens, the focusing lens, the diaphragm mechanism, etc. to the optical system driving unit 32. The optical system driving unit 32 performs movement of the focusing lens and the zoom lens, opening and closing of the diaphragm blades of the diaphragm mechanism, etc. according to these driving instructions.

[0160] In addition, the control unit 33 controls writing of various data to the memory unit 34 and reading of various data from the memory unit 34. For example, the memory unit 34 is a non-volatile storage device such as a hard disk drive (HDD) or a flash memory device, and serves as a storage destination (recording destination) for image data output from the image sensor IS.

[0161] In addition, the control unit 33 performs various types of data communication with an external device via the communication unit 35. The communication unit 35 in this example is configured to be able to perform data communication at least with Figure 1 the cloud server 1 shown in.

[0162] For example, the image sensor IS is configured as a CCD or CMOS image sensor.

[0163] The image sensor IS includes an imaging unit 41, an image signal processing unit 42, an in-sensor control unit 43, an AI image processing unit 44, a memory unit 45, and a communication I / F 46, and these units can perform data communication with each other via a bus 47.

[0164] The imaging unit 41 includes a pixel array unit and a reading circuit. In the pixel array unit, pixels each having a photoelectric conversion element (such as a photodiode) are arranged two-dimensionally. The reading circuit reads an electrical signal obtained by photoelectric conversion from each pixel included in the pixel array unit and can output the electrical signal as a captured image signal.

[0165] The reading circuit performs, for example, correlated double sampling (CDS) processing, automatic gain control (AGC) processing, etc. on the electrical signal obtained by photoelectric conversion, and further performs analog / digital (A / D) conversion processing.

[0166] The image signal processing unit 42 performs preprocessing, synchronization processing, YC generation processing, resolution conversion processing, codec processing, etc. on the captured image signal as digital data after the A / D conversion processing. In the preprocessing, clamping processing for clamping the black levels of R, G, and B to a predetermined level, correction processing between the color channels of R, G, and B, etc. are performed on the captured image signal. In the synchronization processing, color separation processing is performed so that the image data of each pixel has all the R, G, and B color components. For example, in the case of an imaging element using a Bayer array color filter, demosaicking processing is performed as the color separation processing. In the YC generation processing, a luminance (Y) signal and a color (C) signal are generated (separated) from the R, G, and B image data. In the resolution conversion processing, resolution conversion processing is performed on the image data that has undergone various types of signal processing.

[0167] In the codec processing, for example, encoding processing for recording or communication and file generation are performed on the image data that has undergone the above various types of processing. In the codec processing, file generation can be performed in a format such as Moving Picture Experts Group (MPEG)-2 or H.264 as a moving image file format. In addition, it is also conceivable to perform file generation in a format such as Joint Photographic Experts Group (JPEG), Tagged Image File Format (TIFF), or Graphics Interchange Format (GIF) as a still image file.

[0168] The in-sensor control unit 43 executes execution control of the imaging operation by issuing instructions to the imaging unit 41. Similarly, execution control of the processing is also performed on the image signal processing unit 42.

[0169] The AI image processing unit 44 performs image recognition processing on the captured image as AI image processing.

[0170] The image recognition function using AI can be implemented using a programmable arithmetic processing device such as a CPU, a field programmable gate array (FPGA), or a digital signal processor (DSP).

[0171] The image recognition function that can be implemented by the AI image processing unit 44 can be switched by changing the algorithm of the AI image processing. In other words, the type of function of the AI image processing can be switched by switching the AI model used for the AI image processing. Various types of functions of the AI image processing can be considered, and for example, the types illustrated below can be cited as examples.

[0172] · Class recognition

[0173] · Semantic segmentation

[0174] · Person detection

[0175] · Vehicle detection

[0176] · Object tracking

[0177] · Optical character recognition (OCR)

[0178] Among the above function types, class recognition is a function for recognizing the class of an object. Here, "class" is information indicating the object class, and for example, it distinguishes "person", "car", "airplane", "ship", "truck", "bird", "cat", "dog", "deer", "frog", "horse", etc. Object tracking is a function for tracking a subject as an object, and can be called a function for obtaining historical information on the position of the subject.

[0179] The memory unit 45 is used as a storage destination for various data such as the captured image data obtained by the image signal processing unit 42. In addition, in this example, the memory unit 45 can also be used to temporarily store the data used by the AI image processing unit 44 during the AI image processing.

[0180] In addition, the memory unit 45 stores information on the AI application or the AI model used in the AI image processing unit 44.

[0181] Note that the information on the AI application and the AI model can be deployed in the memory unit 45 as a container or the like using the container technology described later, or can be deployed using the microservices technology. By deploying the AI model used for the AI image processing in the memory unit 45, the type of function of the AI image processing can be changed by re-learning or the AI model can be changed to an AI model with improved performance.

[0182] It should be noted that, as described above, in this embodiment, the description is made based on examples of an AI model and an AI application for image recognition. However, the present invention is not limited thereto, and it can be applied to programs executed using AI technology or the like.

[0183] In addition, when the capacity of the memory unit 45 is small, the information of the AI application and the AI model can be deployed in a memory outside the image sensor IS (e.g., the memory unit 34) as a container or the like by using container technology. Then, only the AI model can be stored in the memory unit 45 in the image sensor IS via the communication I / F 46 described below.

[0184] The communication I / F 46 is an interface for communicating with a control unit 33, a memory unit 34, etc. outside the image sensor IS. The communication I / F 46 performs communication from the outside for acquiring a program executed by the image signal processing unit 42, an AI application or an AI model used by the AI image processing unit 44, etc., and stores the program, the AI application, the AI model, etc. in the memory unit 45 included in the image sensor 1S. As a result, the AI model is stored in a part of the memory unit 45 included in the image sensor IS and can be used by the AI image processing unit 44.

[0185] The AI image processing unit 44 performs predetermined image recognition processing by using the AI application or the AI model obtained in this way and performs recognition according to the target object.

[0186] The recognition result information of the AI image processing is output to the outside of the image sensor IS via the communication I / F 46.

[0187] That is, not only the image data output from the image signal processing unit 42 but also the recognition result information of the AI image processing is output from the communication I / F 46 of the image sensor IS. Note that only one of the image data and the recognition result information can be output from the communication I / F 46 of the image sensor IS.

[0188] For example, in the case of using the relearning function of the above AI model, the captured image data for the relearning function is uploaded from the image sensor IS to the cloud-side information processing device via the communication I / F 46 and the communication unit 35.

[0189] In addition, in the case of performing inference using the AI model, the recognition result information of the AI image processing is output from the image sensor IS to other information processing devices outside the camera 3 via the communication I / F 46 and the communication unit 35.

[0190] Various configurations of the image sensor IS can be considered. Here, an example in which the image sensor IS has a structure in which two layers are stacked will be described.

[0191] As shown Figure 8 in FIG. 1, the image sensor IS is configured as a monolithic semiconductor device that stacks two wafers.

[0192] The image sensor IS has a configuration in which a die D1 having the function of an imaging unit 41 ( Figure 7 ) and a die D2 including an image signal processing unit 42, an in-sensor control unit 43, an AI image processing unit 44, a memory unit 45, and a communication I / F 46 are stacked.

[0193] The die D1 and the die D2 are electrically connected, for example, by Cu-Cu bonding.

[0194] Various methods of deploying an AI model, an AI application, etc. in the camera 3 can be considered. As an example, an example using container technology will be described.

[0195] As shown Figure 9 in FIG. 2, in the camera 3, an operating system 51 is installed on various types of hardware 50, such as a CPU, a graphics processing unit (GPU), a ROM, and a RAM, as a control unit 33 ( Figure 7 ).

[0196] The operating system 51 is basic software that executes overall control of the camera 3 to implement various functions in the camera 3.

[0197] General middleware 52 is installed on the operating system 51.

[0198] The general middleware 52 is software for implementing basic operations such as a communication function using the communication unit 35 as the hardware 50 and a display function using the display unit (monitor, etc.) as the hardware 50, for example.

[0199] On the operating system 51, not only the general middleware 52 but also an orchestration tool 53 and a container engine 54 are installed.

[0200] The orchestration tool 53 and the container engine 54 deploy and execute the container 55 by building a cluster 56 as an operating environment for the container 55. Note that Figure 5 the edge runtime shown in FIG. 3 corresponds to Figure 9 the orchestration tool 53 and the container engine 54 shown in FIG. 2.

[0201] The orchestration tool 53 has a function of appropriately allocating the resources of the above-described hardware 50 and the operating system 51 to the container engine 54. Each container 55 is placed in a predetermined unit (a container to be described later) by the orchestration tool 53, and each container is deployed to a worker node (to be described later) that is a logically different area.

[0202] The container engine 54 is one of the middleware installed on the operating system 51 and is an engine for operating the container 55. Specifically, the container engine 54 has a function of allocating resources (memory, operation ability, etc.) of the hardware 50 and the operating system 51 to the container 55 based on a setting file or the like included in the middleware included in the container 55.

[0203] In addition, the resources allocated in the present embodiment include not only resources such as the control unit 33 included in the camera 3, but also resources such as the in-sensor control unit 43, the memory unit 45, and the communication I / F 46 included in the image sensor IS.

[0204] The container 55 includes an application for implementing a predetermined function and middleware such as a library. The container 55 is operated to implement a predetermined function using the resources of the hardware 50 and the operating system 51 allocated by the container engine 54.

[0205] In the present embodiment, Figure 5 The AI application and the AI model shown correspond to one of the containers 55. That is, one of the various containers 55 deployed in the camera 3 uses the AI application and the AI model to implement a predetermined AI image processing function.

[0206] Reference will be made to Figure 10 Describe a specific configuration example of the cluster 56 constructed by the container engine 54 and the orchestration tool 53. Note that the cluster 56 can be constructed across multiple devices so that functions are implemented using not only the resources of the hardware 50 included in one camera 3 but also the resources of other hardware included in other devices.

[0207] The orchestration tool 53 manages the execution environment of the container 55 in units of worker nodes 57. In addition, the coordination tool 53 constructs a master node 58 that manages the entire worker node 57.

[0208] In the worker node 57, a plurality of pods 59 are deployed. The pod 59 includes one or more containers 55 and implements a predetermined function. The pod 59 is a management unit for managing the container 55 by the orchestration tool 53.

[0209] The operation of the pod 59 in the worker node 57 is controlled by the container management library 60.

[0210] The container management library 60 includes a container runtime for enabling the pod 59 to use the logically allocated resources of the hardware 50, an agent that accepts control from the master node 58, a network agent that executes communication between the pods 59 and communication with the master node 58, and the like. That is, each pod 59 can use each resource through the container management library 60 to implement a predetermined function.

[0211] The main node 58 includes: an application server 61, a deployed pod 59; a manager 62 that manages the deployment of the application server 61 to the container 55; a scheduler 63 that determines the worker node 57 in which the container 55 is arranged; and a data sharing unit 64 that performs data sharing.

[0212] By using Figure 9 and Figure 10 the configurations shown in, the above AI application and AI model can be deployed in the image sensor IS of the camera 3 by using container technology. It should be noted that, as described above, the AI model can be stored in the memory unit 45 in the image sensor IS via the Figure 7 communication I / F 46 in, and the AI image processing can be performed in the image sensor IS. Alternatively, the configurations shown in Figure 9 and Figure 10 can be deployed in the memory unit 45 and the in-sensor control unit 43 in the image sensor IS, and the above AI application and AI model can be performed in the image sensor IS by using container technology.

[0213] In addition, even when the AI application and / or AI model is deployed in the fog server or the cloud-side information processing device, container technology can also be used. At this time, the information of the AI application or AI model is deployed and executed in the memory as a container or the like. For example, in the non-volatile memory unit 74, storage unit 79, or RAM 73 described later in Figure 11 .

[0214] <1-5. Hardware Configuration of Information Processing Device>

[0215] The hardware configuration of the information processing devices such as the cloud server 1, user terminal 2, and management server 4 included in the information processing system 100 will be described with reference to Figure 11 .

[0216] The information processing device includes a CPU 71. The CPU 71 serves as an arithmetic processing unit that executes the above various types of processing, and executes various types of processing according to a program stored in a non-volatile memory unit 74 such as a ROM 72 or an electrically erasable programmable read-only memory (EEP-ROM) or a program loaded from a storage unit 79 to a RAM 73. The RAM 73 also appropriately stores data and the like required for the CPU 71 to execute various types of processing.

[0217] Note that the CPU 71 included in the information processing device serving as the cloud server 1 serves as the above-described account service unit 11, device monitoring unit 12, market unit 13, and camera service unit 14. In addition, the CPU 71 included in the information processing device serving as the management server 4 functions as the above-described permission authorization unit 21.

[0218] The CPU 71, ROM 72, RAM 73, and non-volatile memory unit 74 are connected to each other via a bus 83. An input / output interface (I / F) 75 is also connected to the bus 83.

[0219] An input unit 76 composed of an operator or an operating device is connected to the input / output interface 75. For example, various types of operators or operating devices (such as a keyboard, a mouse, a button, a dial, a touch panel, a touchpad, and a remote control) can be considered as the input unit 76. The operation of the user is detected by the input unit 76, and the signal according to the input operation is analyzed by the CPU 71.

[0220] In addition, a display unit 77 composed of an LCD, an organic EL panel, etc., and a sound output unit 78 composed of a speaker, etc. are connected to the input / output interface 75 as a single body or separately. The display unit 77 is a display unit that provides various displays, and is configured with, for example, a display device provided in the housing of the information processing device or a separate display device connected to the information processing device.

[0221] The display unit 77 displays an image for various types of image processing and a moving image to be processed on the display screen in response to an instruction from the CPU 71. The display unit 77 also provides a display as a graphical user interface (GUI) in response to an instruction from the CPU 71, for example, various operation menus, icons, and messages.

[0222] In some cases, a storage unit 79 including a hard disk and a solid-state memory or a communication unit 80 including a modem is connected to the input / output interface 75. It should be noted that the storage unit 79 included in the information processing device as the management server 4 serves as the above-mentioned memory 22.

[0223] The communication unit 80 performs communication processing through a transmission path (such as the Internet), communicates with various devices (such as wired / wireless communication or bus communication), etc.

[0224] As needed, a drive 81 is also connected to the input / output interface 75, and a removable medium 82, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is appropriately installed in the drive 81.

[0225] The drive 81 can be used to read data files such as programs for each example to be processed from the removable medium 82. The read data files are stored in the storage unit 79, and the images or sounds included in the data files are output to the display unit 77 or the sound output unit 78, etc. In addition, a computer program, etc. read from the removable medium 82 is installed in the storage unit 79 as needed.

[0226] In an information processing apparatus, for example, software for processing in the present embodiment can be installed by using network communication of a communication unit 80 or through a removable medium 82. Alternatively, the software can be pre-stored in a ROM 72, a storage unit 79, or the like. In addition, a captured image captured by a camera 3 or a processing result of AI image processing can be received via the storage unit 79 or a drive 81 and stored in the removable medium 82.

[0227] The CPU 71 performs processing operations based on various programs, and performs necessary information processing and communication processing as the cloud server 1, that is, an information processing apparatus including the above-described arithmetic processing unit.

[0228] It should be noted that the cloud server 1 is not limited to having a single information processing apparatus as shown in Figure 11 and may have a configuration in which a plurality of information processing apparatuses are systematized. The plurality of information processing apparatuses can be systematized through a local area network (LAN) or the like, or can be arranged at remote locations through a virtual private network (VPN) using the Internet or the like. The plurality of information processing apparatuses may include information processing apparatuses as a server group (cloud) that can be used by cloud computing services.

[0229] <2. Regarding Camera Settings>

[0230] Figure 12 It is a diagram showing the configuration of each apparatus related to the settings of the camera 3.

[0231] As Figure 12 shown, the settings of the camera 3 are realized by applying the account service unit 11, the permission authorization unit 21, and the memory 22 of the user terminal 2B, the camera 3, and the cloud-side information processing apparatus.

[0232] The camera 3 is provided with an application processor (AP) 91. Figure 12 The AP 91 shown in Figure 7 corresponds to, for example, the control unit 33 shown in

[0233] The AP 91 can communicate with the permission authorization unit 21 and the memory, and the application user terminal 2B can communicate with the account service unit 11.

[0234] The detailed process of the settings of the camera 3 will be described with reference to Figures 13 to 15 (step S51 in Figure 4 ).

[0235] First, as shown by the arrow #1 in Figure 13 , the account service unit 11 generates a two-dimensional code C1 for setting the camera 3 and provides the two-dimensional code C1 to the user. The two-dimensional code C1 stores, for example, connection destination information and network information.

[0236] In the generation of a two-dimensional code, the application user terminal 2B accesses the URL prepared for each user or uses the user's account information to perform a login, thereby connecting to the account service unit 11. The account service unit 11 provides a two-dimensional code creation screen to the application user terminal 2B and accepts the input of information stored in the two-dimensional code. For example, the two-dimensional code creation screen is presented to the user through the display unit of the application user terminal 2B.

[0237] Figure 14 It is a diagram showing an example of the two-dimensional code creation screen.

[0238] The two-dimensional code generation screen has an input field for inputting the connection destination (SETUP URL) of the camera 3, an input field for inputting the network name (WIFISSID) of the Wi-Fi (registered trademark) connection, and an input field for inputting the password of the Wi-Fi connection. In addition, the two-dimensional code creation screen has an input field for inputting the proxy address (PROXY URL proxy URL), an input field for inputting the port number of the proxy server (PROXY PORT proxy port), an input field for inputting the user name (PROXY USERNAME proxy user name) for proxy connection, and an input field for inputting the password (PROXY PASSWORD proxy password) for proxy connection. In addition, the two-dimensional code creation screen is provided with an input field for inputting the Internet Protocol (IP) address, an input field for inputting the subnet mask, an input field for inputting the setting of the Domain Name System (DNS) server, and an input field for inputting the setting of the Network Time Protocol (NTP) server.

[0239] As the connection destination of the camera 3, a URL for accessing the access permission authorization unit 21 is input through the account service unit 11. The URL of the access permission authorization unit 21 is, for example, a user-specific URL.

[0240] The network information includes the network name of the Wi-Fi connection, the password of the Wi-Fi connection, the proxy address, the port number of the proxy server, the user name for proxy connection, the password for proxy connection, the IP address, the subnet mask, the setting of DNS, and the setting of NTP. The user inputs this network information on the two-dimensional code production screen. Note that the proxy address, the port number of the proxy server, the user name for proxy connection, the password for proxy connection, the IP address, the subnet mask, the setting of DNS, and the setting of NTP are information input additionally as options.

[0241] When the camera 3 is compatible with PoE (Power over Ethernet), the user does not need to input network information.

[0242] The button 101 for generating a QR code is set at the lower side of the QR code generation screen. When the button 101 is pressed after inputting network information, the account service unit 11 performs a process of generating a QR code, in which the connection destination information and the network information are stored in the payload. Note that the payload of the QR code can be encrypted information or plain text. The generated QR code C1 is displayed on the right side of the QR code generation screen.

[0243] Below the display area of the QR code C1, there is a button 102 for downloading the QR code. When the button 102 is pressed after the QR code C1 is displayed, the application user terminal 2B downloads the QR code C1 generated by the account service unit 11.

[0244] Next, the application user terminal 2B prints the downloaded QR code C1 or displays the downloaded QR code C1 on the display unit. For example, the user arranges the display unit of the application user terminal 2B at the focal position of the camera 3 to adjust the focus of the camera 3 to the QR code C1. In this state, as Figure 15 shown by the dashed line in, the camera 3 (image sensor IS) images the QR code C1 and acquires a QR code image, which is an image where the QR code appears.

[0245] The AP 91 functions as an acquisition unit that acquires the connection destination information and the network information by decoding (analyzing) the QR code appearing in the QR code image. The AP 91 stores the acquired network information in a flash memory (flash memory device) or the like. Note that the network information can be stored in the flash memory not after acquiring the network information from the QR code but after the settings of the camera 3 are completed.

[0246] The method of acquiring the connection destination information etc. by the camera 3 is not limited to the method using the QR code, and a method of acquiring the connection destination information etc. via wireless communication such as Wi-Fi or Bluetooth (registered trademark) can be considered.

[0247] Next, the AP 91 uses the network information to connect to the network 5 and accesses the URL indicated by the connection destination information to connect to the permission authorization unit 21. The AP 91 transmits the device information inherent to the camera 3 to the permission authorization unit 21.

[0248] The permission authorization unit 21 receives the device information transmitted from the AP 91. The permission authorization unit 21 functions as an authentication unit that authenticates the camera 3 by comparing the received device information (first device information) with the pre-registered device information (second device information).

[0249] When the first device information matches the second device information, the license authorization unit 21 obtains the URL for downloading the FW from the memory 22. For example, a shared access signature (SAS) URL with at least one of an expiration date and access restrictions set therein is obtained as the URL for downloading the FW. The license authorization unit 21 transmits the obtained SAS URL to the AP 91.

[0250] The AP 91 receives the SAS URL transmitted by the license authorization unit 21, accesses the SAS URL, and downloads the FW from the memory 22. As described above, the AP 91 also serves as a communication control unit, which transmits the device information unique to the camera 3 to the license authorization unit 21 and downloads the FW from the memory 22. The AP 91 updates the downloaded FW to complete the settings of the camera 3.

[0251] Since the initial FW having only the functions of analyzing the QR code and connecting to the license authorization unit 21 and the memory 22 is incorporated in the AP 91 (camera 3) at the time of shipment from the factory, the camera 3 cannot perform AI image processing or the like. Since AI image processing or the like is not performed, the user cannot use the camera 3 to operate the application of the service. When the authentication of the camera 3 is successful and the initial FW incorporated in the AP 91 is updated to the FW downloaded from the memory 22, the camera 3 can perform AI image processing or the like.

[0252] Figure 16 is a diagram showing an example of the authentication process of the camera 3.

[0253] Figure 16 shows an example in which the camera 3 is connected to different clouds of each user to perform settings.

[0254] As Figure 16 shown in the upper part of, it is assumed that user A purchased three cameras 3-1 to 3-3. As shown in dialog boxes #21 to #23, at the time of shipment from the factory, the device information unique to each device is stored in each of the cameras 3-1 to 3-3.

[0255] User A accesses cloud A through the application user terminal 2B and obtains the QR code C11. Cloud A is a cloud-side information processing device dedicated to user A, which provides services to user A. In the QR code C11, as shown in dialog box #24, the connection destination URL (connection destination information) for connecting to cloud A and network information are stored.

[0256] The cameras 3-1 to 3-3 connect to cloud A based on the connection destination information obtained from the QR code C11 and transmit the device information stored therein to cloud A.

[0257] When cameras 3-1 to 3-3 are purchased, as shown in dialog box #25, the device information of each of cameras 3-1 to 3-3 is registered in Cloud A. In other words, cameras 3-1 to 3-3 are registered in association with User A. Cloud A compares the device information transmitted from each of cameras 3-1 to 3-3 with the registered device information. When the device information transmitted from each of cameras 3-1 to 3-3 matches the registered device information, the authentication of cameras 3-1 to 3-3 is successful, and cameras 3-1 to 3-3 can update the FW.

[0258] As Figure 16 shown in the lower part of, it is assumed that User B purchases two cameras 3-4 and 3-5. As shown in dialog boxes #31 and #32, at the time of shipment from the factory, device information unique to each device is stored in each of cameras 3-4 and 3-5.

[0259] User B accesses Cloud B through the application user terminal 2B and obtains the QR code C12. Cloud B is a cloud-side information processing device dedicated to User B, which provides services to User B. In the QR code C12, as shown in dialog box #33, the connection destination URL (connection destination information) for connecting to Cloud B and network information are stored.

[0260] Cameras 3-4 and 3-5 are connected to Cloud B based on the connection destination information obtained from the QR code C12, and transmit the device information stored in Cloud B to Cloud B.

[0261] When cameras 3-4 and 3-5 are purchased, as shown in dialog box #34, the device information of each of cameras 3-4 and 3-5 is registered in Cloud B. In other words, cameras 3-4 and 3-5 are registered in association with User B. Cloud B compares the device information transmitted from each of cameras 3-4 and 3-5 with the registered device information. When the device information transmitted from each of cameras 3-4 and 3-5 matches the registered device information, the authentication of cameras 3-4 and 3-5 is successful, and cameras 3-4 and 3-5 can update the FW.

[0262] For example, even if User B makes camera 3-1 photograph the QR code C12 storing the connection destination URL for connecting to Cloud B and camera 3-1 is connected to Cloud B, the device information of camera 3-1 is not registered in Cloud B, so camera 3-1 is not authenticated. Without performing authentication, the FW cannot be updated, and thus camera 3-1 cannot perform AI image processing, etc. Therefore, User B cannot use camera 3-1 purchased by User A.

[0263] As described above, unless an authorized user uses the QR code obtained through the authorization process to set up camera 3, various functions of camera 3 cannot be used. Therefore, only the authorized user who purchases camera 3 can use camera 3.

[0264] Next, the processing performed by the edge-side information processing device (Camera 3) and the cloud-side information processing device (Cloud Server 1 and Management Server 4) for setting up Camera 3 will be described with reference to the sequence diagram in Figure 17 .

[0265] In step S101, the user accesses the account service unit 11 of the cloud-side information processing device by using the application user terminal 2B and generates a QR code.

[0266] The account service unit 11 generates a QR code based on the network information and the like input by the user on the QR code creation screen in step S41, and causes the application user terminal 2B to download the QR code in step S42.

[0267] In step S102, the application user terminal 2B downloads the QR code from the account service unit 11.

[0268] In step S103, the user presents the QR code to Camera 3 by, for example, arranging the display unit of the application user terminal 2B on which the QR code is displayed at the focal position of Camera 3.

[0269] In step S111, the image sensor IS of Camera 3 images the QR code and acquires a QR code image.

[0270] In step S112, the image sensor IS provides the QR code image to the AP 91 of Camera 3.

[0271] In step S121, the AP 91 acquires the QR code image provided from the image sensor IS.

[0272] In step S122, the AP 91 performs an identification process (analysis) on the QR code shown in the QR code image and acquires connection destination information and network information.

[0273] In step S123, the AP 91 stores the network information obtained from the QR code in the flash memory.

[0274] In step S124, the AP 91 connects to Network � based on the network information and connects to the permission authorization unit 21 of the edge-side information processing device based on the connection destination information. After the AP 91 connects to the permission authorization unit 21, it transmits the device information unique to Camera 3 to the permission authorization unit 21.

[0275] In step S131, the permission authorization unit 21 receives the device information transmitted from the AP 91, and in step S132, compares the device information transmitted from the AP 91 with the pre-registered device information.

[0276] In the case where it is determined that the device information transmitted from the AP 91 is not pre-registered device information, in step S133, the permission and authorization unit 21 rejects the connection of the AP 91.

[0277] In step S134, in the case where it is determined that the device information transmitted from the AP 91 is pre-registered device information, the permission and authorization unit 21 provides a request for obtaining the SAS URL for the camera 3 to download the FW to the memory 22 of the edge-side information processing device.

[0278] In step S151, when the memory 22 receives a request for obtaining the SAS URL from the permission and authorization unit 21, in step S152, the memory 22 provides the SAS URL to the permission and authorization unit 21.

[0279] The permission and authorization unit 21 obtains the SAS URL supplied from the memory 22 in step S135 and transmits the SAS URL to the AP 91 in step S136.

[0280] In step S125, the AP 91 receives the SAS URL transmitted from the permission and authorization unit 21 and connects to the memory 22 based on the SAS URL.

[0281] In step S126, the AP 91 transmits a download request for the FW to the memory 22.

[0282] In the case where the memory 22 receives a download request for the FW transmitted from the AP 91 in step S153, the memory 22 enables the AP 91 to download the FW in step S154.

[0283] In step S127, the AP 91 downloads the FW from the memory 22 and updates the initial FW to the FW downloaded from the memory 22 in step S128.

[0284] As described above, in the cloud-side information processing device, device authentication is performed by comparing the first device information unique to the device such as the camera 3 transmitted from the camera 3 that has acquired the connection destination information indicating the connection destination with the pre-registered second device information. In addition, in the edge-side information processing device, the connection destination information of the cloud-side information processing device that represents the execution of device authentication as the connection destination is acquired, the first device information is transmitted to the cloud-side information processing device indicated by the connection destination information, and in the case of successful device authentication (in the case where the set execution is authorized), the FW is downloaded from the cloud-side information processing device. In the case of successful device authentication, that is, in the case where an authorized user is about to perform the device settings, the device settings are completed, and only the authorized user can use the device.

[0285] Note that multiple cameras 3 used by multiple users can be connected to a cloud to perform settings.

[0286] Figure 18 It is a diagram showing an example of another process for authenticating the camera 3.

[0287] Similar to the example Figure 16 described, assume that user A has purchased three cameras 3-1 to 3-3, and user B has purchased two cameras 3-4 and 3-5.

[0288] User A accesses the cloud through the application user terminal 2B and obtains the QR code C21. The cloud is a cloud-side information processing device that provides services to multiple users such as user A and user B. In the QR code C21, as shown in dialog box #41, the connection destination URL (connection destination information) for connecting to the cloud, the account information of user A, and network information are stored.

[0289] Cameras 3-1 to 3-3 are connected to the cloud based on the connection destination information obtained from the QR code C21, and transmit the device information stored therein and the account information of user A obtained from the QR code C21 to the cloud.

[0290] When purchasing cameras 3-1 to 3-3, as shown in dialog box #43, the device information of each of cameras 3-1 to 3-3 is registered in the cloud in association with the account information of user A. The cloud compares the device information transmitted from each of cameras 3-1 to 3-3 with the device information registered in association with the account information of user A. When the device information transmitted from each of cameras 3-1 to 3-3 matches the registered device information, the authentication of cameras 3-1 to 3-3 is successful, and cameras 3-1 to 3-3 can update the FW.

[0291] On the other hand, user B accesses the cloud through the application user terminal 2B and obtains the QR code C22. In the QR code C22, as shown in dialog box #42, the connection destination URL (connection destination information) for connecting to the cloud, the account information of user B, and network information are stored.

[0292] Cameras 3-4, 3-5 are connected to the cloud based on the connection destination information obtained from the QR code C22, and transmit the device information stored therein and the account information of user B obtained from the QR code C22 to the cloud.

[0293] When cameras 3-4 and 3-5 are purchased, as shown in dialog box #43, the device information of each of cameras 3-4 and 3-5 is registered in the cloud in association with the account information of user B. Cloud B compares the device information transmitted from each of cameras 3-4 and 3-5 with the device information registered in association with the account information of user B. In the case where the device information transmitted from each of cameras 3-4 and 3-5 matches the registered device information, the authentication of cameras 3-4 and 3-5 is successful, and cameras 3-4 and 3-5 can update the FW.

[0294] For example, even when user B uses camera 3-1 to photograph the two-dimensional code C22 storing the account information of user B and camera 3-1 is connected to the cloud, the device information of camera 3-1 is not registered in association with the account information of user B, so camera 3-1 is not authenticated. In the case where authentication is not performed, the FW cannot be updated, and thus camera 3-1 cannot perform AI image processing, etc. Therefore, user B cannot use camera 3-1 purchased by user A.

[0295] As described above, even when multiple cameras 3 used by multiple users are connected to one cloud to perform settings, only the authorized user who purchases the camera 3 can use the camera 3.

[0296] Note that when a camera 3 is purchased, only the device information can be registered in the cloud without registering the user's account information. In the case where an unpurchased camera 3 is stolen, since the device information of the stolen camera 3 is not registered in the cloud, even when an unauthorized user who has not purchased the camera 3 attempts to perform the settings of the stolen camera 3, the stolen camera 3 is not authenticated. In the case where authentication is not performed, the FW cannot be updated, and thus the stolen camera 3 cannot perform AI image processing, etc. Therefore, the unauthorized user cannot use the stolen camera 3.

[0297] <3. Regarding relearning>

[0298] As described above, it will be described with reference to Figure 19 Specifically, the operations of the service provider or user (user) who utilizes the service after deploying the SW components of the AI application and the AI model are described as a trigger to execute the relearning of the AI model and the update of the AI model (hereinafter, referred to as "edge-side AI model") or the process when deploying the AI application to each camera 3, etc. It should be noted that Figure 19 Focus on one of the multiple cameras 3. In addition, in the following description, the edge-side AI model to be updated is deployed in the image sensor IS included in the camera 3 as an example, but of course, the edge-side AI model can be deployed outside the image sensor IS in the camera 3.

[0299] First, in processing step PS1, the service provider or the user of the service issues a relearning instruction for the AI model. This instruction is executed using an application programming interface (API) function of the API module included in the information processing device on the cloud side. Additionally, in the instruction, the amount of images for learning (e.g., the number of images) is specified. Hereinafter, the amount of images for learning is also referred to as the "predetermined number of images".

[0300] In response to this instruction, the API module transmits a relearning request and image amount information to the Hub (similar to the Hub shown in Figure 5 ).

[0301] In processing step PS3, the Hub transmits an update notification and image amount information to the camera 3, which is an edge-side information processing device.

[0302] The camera 3 transmits the captured image data obtained by performing imaging to the image database (DB) of the storage management unit in processing step PS4. Imaging processing and transmission processing are performed until the predetermined number of images required for relearning is achieved.

[0303] Note that in the case where the camera 3 obtains an inference result by performing inference processing on the captured image data, the inference result can be stored in the image DB as metadata of the captured image data in processing step PS4.

[0304] Since the inference result in the camera 3 is stored in the image DB as metadata, the data required for the relearning of the AI model executed on the cloud side can be carefully selected. Specifically, relearning can be performed using only the image data for which the inference result in the camera 3 is different from the inference result obtained by using the rich computing resources in the cloud-side information processing device. Therefore, the time required for relearning can be shortened.

[0305] After the shooting and transmission of the predetermined number of images are completed, in processing step PS5, the camera 3 notifies the Hub that the transmission of the predetermined number of captured image data is completed.

[0306] When receiving the notification, the hub notifies the orchestration tool in processing step PS6 that the preparation of the data for relearning is completed.

[0307] In processing step PS7, the orchestration tool transmits an execution instruction for the marking process to the marking module.

[0308] The marking module acquires the image data targeted for the marking process from the image DB (processing step PS8) and performs the marking process.

[0309] The tag processing described in this document can be a process of performing the above category recognition, a process of estimating the gender and age of an object in an image and assigning tags, a process of estimating the pose of an object and assigning tags, or a process of estimating the behavior of an object and assigning tags.

[0310] The tag processing can be performed manually or automatically. Additionally, the tag processing can be completed by a cloud-side information processing device or can be implemented by using services provided by other server devices.

[0311] The tag module that has completed the tag processing stores the tag result information in the data set DB in processing step PS9. Here, the information stored in the data set DB can be a collection of tag information and image data, or can be image recognition (ID) information for specifying the image data instead of the image data itself.

[0312] In processing step PS10, the storage management unit that has detected the storage of the tag result information notifies the orchestration tool.

[0313] The orchestration tool that has received the notification checks that the tag processing of a predetermined number of image data has ended and transmits a re-learning instruction to the re-learning module in processing step PS11.

[0314] The re-learning module that has received the re-learning instruction obtains the data set for learning from the data set DB in processing step PS12 and obtains the AI model as the update target from the trained AI model DB in processing step PS13.

[0315] The re-learning module re-learns the AI model by using the obtained data set and AI model. In processing step PS14, the updated AI model obtained in this way is stored again in the trained AI model DB.

[0316] In processing step PS15, the storage management unit that has detected the storage of the updated AI model notifies the orchestration tool.

[0317] In processing step PS16, the orchestration tool that has received the notification transmits a conversion instruction of the AI model to the conversion module.

[0318] In processing step PS17, the conversion module that has received the conversion instruction obtains the updated AI model from the trained AI model DB and performs the conversion process of the AI model. In the conversion process, a process of converting according to the specification information of the camera 3 of the device as the deployment destination, etc. is performed. In this process, miniaturization is performed so as not to deteriorate the performance of the AI model as much as possible, and conversion of the file format, etc. is performed to be operable on the camera 3.

[0319] The AI model converted by the conversion module is the above-mentioned edge-side AI model. In processing step PS18, the converted AI model is stored in the converted AI model DB.

[0320] In processing step PS19, the storage management unit that detects the storage of the converted AI model notifies the orchestration tool.

[0321] In processing step PS20, the orchestration tool that has received the notification transmits a notification for performing AI model update to the Hub. This notification includes information for specifying the location where the AI model for update is stored.

[0322] In processing step PS21, the Hub that has received the notification transmits an update instruction for the AI model to Camera 3. The update instruction also includes information for specifying the location where the AI model is stored.

[0323] In processing step PS22, Camera 3 performs processing to obtain and expand the target converted AI model from the converted AI model DB. As a result, the AI model used in the image sensor IS of Camera 3 is updated.

[0324] In processing step PS23, Camera 3 that has completed the AI model update by deploying the AI model transmits an update completion notification to the Hub. The Hub that has received this notification notifies the orchestration tool in processing step PS24 that the AI model update process of Camera 3 is completed.

[0325] Note that here, an example of deploying and using an AI model in the image sensor IS of Camera 3 (for example, in the memory unit 45 shown in Figure 7 is described. However, even when the AI model is configured and used outside the image sensor (for example, in the memory unit 34 in Figure 7 or in the fog server in Camera 3, the update of the AI model can be performed similarly.

[0326] In this case, when deploying the AI model, the device (location) where the AI model is deployed is stored in a storage management unit on the cloud side, etc., and the hub reads the device (location) where the AI model is deployed from the storage management unit and transmits an update instruction for the AI model to the device where the AI model is deployed.

[0327] In processing step PS22, the device that has received the update instruction performs processing to obtain and expand the object-converted AI model from the converted AI model DB. Therefore, the AI model of the device that has received the update instruction is updated.

[0328] Note that, in the case of only performing the update of the AI model, the processing up to this point is completed. In the case of performing the update of the AI application using the AI model in addition to the update of the AI model, the processing described below is further performed.

[0329] Specifically, in processing step PS25, the orchestration tool transmits a download instruction of the AI application, such as updated firmware, to the deployment control module.

[0330] In processing step PS26, the deployment control module transmits a deployment instruction of the AI application to the Hub. This instruction includes information for specifying the location where the updated AI application is stored.

[0331] In processing step PS27, the Hub transmits a deployment instruction to Camera 3.

[0332] In processing step PS28, Camera 3 downloads and deploys the updated AI application from the container DB of the deployment control module.

[0333] In addition, in the case where both the AI model and the AI application are to operate in one device, both the AI model and the AI application can be updated together as one container. In this case, the update of the AI model and the update of the AI application can be performed simultaneously instead of sequentially. Then, it can be achieved by performing each of the processing steps PS25, PS26, PS27, and PS28.

[0334] For example, in the case where both the container of the AI model and the container of the AI application can be deployed in the image sensor IS of Camera 3, the update of the AI model and the AI application can be performed by performing each of the processing steps PS25, PS26, PS27, and PS28 as described above.

[0335] By performing the above processing, re-learning of the AI model is performed using the captured image data captured in the user's usage environment. Therefore, an edge-side AI model capable of outputting high-precision recognition results in the user's usage environment can be generated.

[0336] In addition, even when the user's usage environment changes, for example, when the layout in the store changes or when the installation location of Camera 3 changes, re-learning of the AI model can be appropriately performed each time, and therefore, the recognition accuracy of the AI model can be maintained without deterioration. Note that not only when re-learning of the AI model is performed, but also when the system is first operated in the user's usage environment, the above-described various processes can be performed.

[0337] <4. Screen Example of the Market>

[0338] Reference will be made to Figures 20 to 22Describe an example of a screen presenting the market to the user.

[0339] Figure 20 Show an example of the login screen G1.

[0340] The login screen G1 has an ID input area 111 for entering the user ID and a password input area 112 for entering the password.

[0341] Below the password input field 112, there is a login button 113 for performing login and a cancel button 114 for canceling the login.

[0342] In addition, below the buttons, there are operators for switching to a page for users who have forgotten their passwords, operators for switching to a page for new user registration, etc., arranged appropriately.

[0343] When the login button 113 is pressed after entering the appropriate user ID and password, a process of switching to the user-specific page is executed in each of the cloud server 1 and the user terminal 2.

[0344] Figure 21 It is an example of a screen presented to, for example, an AI application developer using the application developer terminal 2A and an AI model developer using the AI model developer terminal 2C.

[0345] Each developer can purchase learning datasets, AI models, and AI applications for development through the market. In addition, an AI application or AI model developed by oneself can be registered in the market.

[0346] In Figure 21 On the developer-oriented screen G2 shown in, learnable datasets, AI models, AI applications, etc. (collectively referred to as "data" hereinafter) are displayed on the left.

[0347] Note that although not shown, when purchasing a learning dataset, an image of the learning dataset is displayed on the monitor, and only the desired part of the image is surrounded by a box using an input device such as a mouse, and a name is entered, thereby enabling preparation for learning.

[0348] For example, in the case where it is desired to perform AI learning using an image of a cat, by surrounding only a part of the cat in the image with a frame and entering "cat" as text input, an image with cat annotations added can be prepared for AI learning.

[0349] In addition, in order to easily find the desired data, purposes such as "traffic monitoring", "flow line analysis", and "customer counting" can be selectable. That is, a display process of only showing data suitable for the selected purpose is executed in each of the cloud server 1 and the user terminal 2.

[0350] Note that the purchase price of each piece of data can be displayed on the developer - facing screen G2.

[0351] In addition, on the right side of the developer - facing screen G2, an input field 115 for registering the learning data sets collected or created by the developer, as well as the AI models or AI applications developed by the developer, is provided.

[0352] Input fields 115 for inputting the name and data storage location are set for each piece of data. In addition, a checkbox 116 for setting the necessity / non - necessity of retraining is set for the AI model.

[0353] Note that a price setting field (described as input field 115 in the drawing) for setting the price required to purchase the data to be registered can be provided.

[0354] In addition, at the upper part of the developer - facing screen G2, the user name, the final login date, etc. are displayed as part of the user information. Note that in addition to this, the amount of currency, points, etc. that can be used when the user purchases data can be displayed.

[0355] Figure 22 An example of the user - facing screen G3 presented to the user (the above - mentioned application - using user) who performs various analyses, etc. by deploying the AI application and the AI model in the camera 3 which is an edge - side information - processing device managed by the user himself / herself is shown.

[0356] The user can purchase the camera 3 arranged in the space as a monitoring target through the market. Therefore, on the left side of the user - facing screen G3, radio buttons 117 for selecting the type and performance of the image sensor IS installed on the camera 3, the performance of the camera 3, etc. are arranged.

[0357] In addition, the user can purchase an information - processing device as a fog server through the market. Therefore, radio buttons 117 for selecting each performance of the fog server are arranged on the left side of the user - facing screen G3. In addition, users who already have a fog server can register the performance of the fog server by inputting the performance information of the fog server here.

[0358] The user realizes the desired function by installing the purchased camera 3 (optionally, the camera 3 purchased without going through the market can be used) at an arbitrary location such as a store managed by the user, and in order to maximize the function of each camera 3, information about the installation location of the camera 3 can be registered in the market.

[0359] On the right side of the user - facing screen G3, radio buttons 118 for selecting the environmental information about the installation of the camera 3 are arranged. The user sets the above - mentioned optimal imaging settings for the target camera 3 by appropriately selecting the environmental information about the installation of the camera 3.

[0360] Note that when purchasing the camera 3 and determining the installation location of the camera 3 to be purchased, by selecting each item on the left side and each item on the right side of the user-facing screen G3, it is possible to purchase the camera 3 with the optimal imaging settings preset according to the installation planned location.

[0361] An execution button 119 is provided on the user-facing screen G3. By pressing the execution button 119, the screen changes to a confirmation screen for confirming the purchase or a confirmation screen for confirming the setting of the environment information. As a result, the user can purchase the desired camera 3 or fog server, and can perform the setting of the environment information of the camera 3.

[0362] In the market, for the case where the installation position of the camera 3 is changed, the environment information of each camera 3 can be changed. By re-entering the environment information related to the installation position of the camera 3 on the change screen (not shown), the optimal imaging settings of the camera 3 can be reset.

[0363] <Others>

[0364] The above series of processes can also be executed by hardware or software. When the series of processes are executed by software, the program constituting the software is installed on a computer configured as dedicated hardware, a general-purpose personal computer, etc.

[0365] By providing the program to be installed by recording it on the removable medium 82 shown in Figure 11 , the removable medium 82 is implemented by an optical disc (compact disc read-only memory (CD-ROM), digital versatile disc (DVD), etc.), a semiconductor memory, etc. In addition, the program can also be provided through a wired or wireless transmission medium (such as a local area network, the Internet, or digital satellite broadcasting). The program can be pre-installed in the ROM 72, the storage unit 79, etc.

[0366] It should be noted that the program executed by the computer can be a program that executes processes in chronological order according to the order described in this specification, or can be a program that executes processes in parallel or at necessary timings such as the call time.

[0367] Note that in this specification, a system means a group of multiple constituent elements (devices, modules (components, etc.)), and all the constituent elements may or may not be located in the same housing. Therefore, multiple devices housed in separate housings and connected by a network, as well as one device in which multiple modules are housed in one housing, are both systems.

[0368] It should be noted that the effects described in this specification are only examples and are not intended to be limiting, and other effects can be obtained.

[0369] Embodiments of the present technology are not limited to the above embodiments, and various changes can be made without departing from the gist of the present technology.

[0370] In addition, each step described in the above flowchart can be executed by one device or sharedly executed by multiple devices.

[0371] In addition, in the case where one step includes multiple types of processing, the multiple types of processing included in the one step can be executed by one device or sharedly executed by multiple devices.

[0372] <Combination examples of configurations>

[0373] The present technology may also have the following configurations. (1)

[0375] An information processing method, comprising:

[0376] Via a server device,

[0377] Performing device authentication by comparing first device information unique to a device transmitted from a device that has acquired connection destination information indicating a connection destination with second device information registered in advance. (2)

[0379] According to the information processing method of (1),

[0380] wherein, in the case where the first device information matches the second device information, information for device firmware download is transmitted to the device. (3)

[0382] According to the information processing method of (2),

[0383] wherein the information for firmware download includes a URL in which at least one of an expiration date and an access restriction is set. (4)

[0385] According to the information processing method of (2) or (3),

[0386] wherein the firmware includes at least a part of a program for the device to execute an AI application. (5)

[0388] According to the information processing method of any one of (1) to (4),

[0389] wherein the second device information is registered in association with the user of the device. (6)

[0391] According to the information processing method of (5),

[0392] Among them, the connection destination information indicates the connection destination for different server devices of each user. (7)

[0394] According to the information processing method of (5),

[0395] Among them, the connection destination information indicates the server device to which multiple devices used by multiple users are connected as the connection destination, and

[0396] The device obtains account information about the user together with the connection destination information. (8)

[0398] According to the information processing method of any one of (1) to (7),

[0399] Among them, a two-dimensional code storing the connection destination information is generated, and the two-dimensional code is provided to the user of the device, and

[0400] The connection destination information is obtained by the device imaging the two-dimensional code with a camera and analyzing the two-dimensional code. (9)

[0402] According to the information processing method of (8),

[0403] Among them, a two-dimensional code creation screen for inputting the connection destination information stored in the two-dimensional code is presented to the user. (10)

[0405] According to the information processing method of (8) or (9),

[0406] Among them, network information for the device to be connected to the network is stored in the two-dimensional code together with the connection destination information. (11)

[0408] A server device includes:

[0409] An authentication unit that authenticates the device by comparing the first device information inherent to the device transmitted from the device that has obtained the connection destination information indicating the connection destination with the second device information registered in advance. (12)

[0411] An information processing method includes:

[0412] Via an information processing device,

[0413] Obtain connection destination information that indicates the following server as the connection destination, and the server authenticates its own device by comparing the first device information inherent to its own device with the second device information registered in advance;

[0414] Transmit the first device information to the server indicated by the connection destination information; and

[0415] When the first device information matches the second device information, download the firmware of its own device from the server. (13)

[0417] According to the information processing method of (12),

[0418] wherein the firmware for obtaining the connection destination information and connecting to the server is pre-included in the information processing device, and

[0419] Perform the setting by updating the firmware to the firmware downloaded from the server. (14)

[0421] According to the information processing method of (12) or (13),

[0422] wherein the firmware downloaded from the server includes at least a part of the firmware for the information processing device to execute the AI application program. (15)

[0424] According to the information processing method of any one of (12) to (14),

[0425] wherein the connection destination information is obtained by imaging the QR code generated by the server using a camera and analyzing the QR code. (16)

[0427] According to the information processing method of (15),

[0428] wherein the network information for connecting its own device to the network is stored in the QR code together with the connection destination information. (17)

[0430] According to the information processing method of any one of (12) to (16),

[0431] wherein the connection destination information indicates the server to which different servers for each user are connected as the connection destination. (18)

[0433] According to the information processing method of any one of (12) to (16),

[0434] wherein the connection destination information indicates the server to which multiple devices used by multiple users are connected as the connection destination, and

[0435] Obtain the account information of the user of the information processing device together with the connection destination information. (19)

[0437] An information processing apparatus, comprising:

[0438] an acquisition unit that acquires connection destination information indicating a server as a connection destination, the server performing authentication of its own device by comparing first device information unique to its own device with second device information registered in advance; and

[0439] a communication control unit that transmits the first device information to the server and downloads firmware for its own device from the server when the first device information matches the second device information. (20)

[0441] The information processing apparatus according to (19), further comprising:

[0442] a sensor that images a two-dimensional code;

[0443] wherein the acquisition unit acquires the connection destination information by analyzing the two-dimensional code shown in a captured image of the sensor.

[0444] [Reference label list]

[0445] 1 Cloud server

[0446] 2 User terminal

[0447] 3 Camera

[0448] 4 Management server

[0449] 5 Network

[0450] 11 Account service unit

[0451] 12 Device monitoring unit

[0452] 13 Market unit

[0453] 14 Camera service unit

[0454] 21 Permission authorization unit

[0455] 22 Memory

[0456] 91AP

[0457] IS image sensor

Claims

1. An information processing method, comprising: Via a server device, Performing authentication of a device by comparing first device information unique to the device, transmitted from a device that has acquired connection destination information indicating a connection destination, with second device information registered in advance.

2. The information processing method according to claim 1, Among them, When the first device information matches the second device information, transmitting information for the device to download firmware to the device.

3. The information processing method according to claim 2, Among them, The information for downloading the firmware includes a URL, in which at least one of an expiration date and an access restriction is set.

4. The information processing method according to claim 2, Among them, The firmware includes at least a part of a program for the device to execute an AI application.

5. The information processing method according to claim 1, Among them, Registering the second device information in association with a user of the device.

6. The information processing method according to claim 5, Among them, The connection destination information indicates the server device different for each user as the connection destination.

7. The information processing method according to claim 5, Among them, The connection destination information indicates the server device to which a plurality of devices used by a plurality of users are connected, and The device acquires account information about the user together with the connection destination information.

8. The information processing method according to claim 1, Among them, Generating a QR code storing the connection destination information and providing the QR code to a user of the device, and The connection destination information is acquired by the device by imaging the QR code with a camera and analyzing the QR code.

9. The information processing method according to claim 8, Among them, Presenting a QR code creation screen for inputting the connection destination information stored in the QR code to the user.

10. The information processing method according to claim 8, Among them, Storing network information for the device to be connected to a network together with the connection destination information in the QR code.

11. A server device, comprising: An authentication unit that performs authentication of a device by comparing first device information unique to the device, transmitted from a device that has acquired connection destination information indicating a connection destination, with second device information registered in advance.

12. An information processing method, comprising: Via an information processing device, Acquiring connection destination information indicating a server as a connection destination, the server performing authentication of its own device by comparing first device information unique to its own device with second device information registered in advance; Transmitting the first device information to the server indicated by the connection destination information; And When the first device information matches the second device information, downloading firmware for its own device from the server.

13. The information processing method according to claim 12, Among them, The firmware for obtaining the connection destination information and connecting to the server is pre - included in the information processing device, and the setting is performed by updating the firmware to the firmware downloaded from the server.

14. The information processing method according to claim 12, Among them, the firmware downloaded from the server includes at least a part of the firmware for the information processing device to execute an AI application program.

15. The information processing method according to claim 12, Among them, the connection destination information is obtained by imaging a two - dimensional code generated by the server using a camera and analyzing the two - dimensional code.

16. The information processing method according to claim 15, Among them, the network information of the own device to be connected to the network is stored in the two - dimensional code together with the connection destination information.

17. The information processing method according to claim 12, Among them, the connection destination information indicates the server different for each user as the connection destination.

18. The information processing method according to claim 12, Among them, the connection destination information indicates the server to which a plurality of devices used by a plurality of users are connected as the connection destination, and the account information of the user of the information processing device is obtained together with the connection destination information.

19. An information processing device, comprising: an acquisition unit that acquires connection destination information indicating a server as a connection destination, the server performing authentication of the own device by comparing the unique first device information of the own device with pre - registered second device information; and a communication control unit that transmits the first device information to the server and, when the first device information matches the second device information, downloads the firmware of the own device from the server.

20. The information processing device according to claim 19, further comprising: a sensor that images a two - dimensional code; wherein, the acquisition unit acquires the connection destination information by analyzing the two - dimensional code shown in the captured image of the sensor.

Images