Information processing apparatus, information processing method, program, and recording medium

Abstract

An information processing system includes: a database configured to store feature data that is generated from sensing data using a backbone model, and processing circuitry configured to select a head model for inference processing; and cause the selected head model to be used for the inference processing using the feature data that is generated from the sensing data using the backbone model.

Description

INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, PROGRAM, AND RECORDING MEDIUM

[0001] The present technique relates to an information processing apparatus, an information processing method, a program, and a recording medium and, in particular, to a technique of performing inference processing using an AI (Artificial Intelligence) model on sensing data from a sensor.

[0002] For example, there is a technique of performing inference processing using an AI (Artificial Intelligence) model on sensing data such as captured image data obtained by a camera and there is also a technique of performing various analyses with respect to a target of sensing based on a result of such inference processing.　For example, conceivably, inference processing such as human detection processing using an AI model is performed on captured image data from a camera installed in a store and, based on an inference result, an analysis such as taking a headcount of customers is performed.

[0003] An example of related conventional art includes PTL 1 described below.　PTL 1 described below discloses a sensor-integrated inference apparatus in which a signal processing unit that performs inference processing is mounted inside an image sensor in which a pixel array portion is formed as an inference apparatus that performs inference processing on image data as target data.

[0004] WO 2023 / 090119Summary

[0005] Let us now consider a situation where output data from a sensing apparatus including a sensor is accumulated in a database and various analyses of a target of sensing are performed based on the accumulated data in the database. In this case, conceivably, sensing data output by a sensing apparatus such as captured image data from a camera is accumulated in the database and a later-stage information processing apparatus is configured to perform inference processing using an AI model on the sensing data and perform analysis processing based on an inference result.

[0006] However, accumulating sensing data in a database as described above is undesirable because it leads to an increase in storage capacity of the database.　In particular, if the sensing data is image data (moving image data), accumulating the sensing data leads to a significant increase in storage capacity. In addition, when sensing data is accumulated in a database, an amount of communication data between the sensing apparatus and the database also tends to increase, which is also undesirable.

[0007] Furthermore, considering the possibility of data leakage between the sensing apparatus and the database, accumulating sensing data in the database is also undesirable from a perspective of privacy protection.　In particular, when the sensing data is image data or sound data, countermeasures need to be taken since a person can be readily identified from leaked data.

[0008] The present technique has been devised in consideration of the circumstances described above and an object thereof is to achieve, when performing inference processing using an AI model based on data accumulated in a database from a sensing apparatus, privacy protection while reducing a storage capacity of the database and an amount of communication data between the sensing apparatus and the database.

[0009] An information processing system according to the present technique includes: a database configured to store feature data that is generated from sensing data using a backbone model, and processing circuitry configured to select a head model for inference processing; and cause the selected head model to be used for the inference processing using the feature data that is generated from the sensing data using the backbone model.

[0010] Fig. 1 is a block diagram showing a schematic configuration example of an inference system as a first embodiment according to the present technique.Fig. 2 is a block diagram showing a configuration example of a sensing apparatus in the embodiment.Fig. 3 is a block diagram showing an example of a hardware configuration of an information processing apparatus as the embodiment.Fig. 4 is an explanatory diagram with respect to functional partitioning of an AI model that is premised in the embodiment.Fig. 5 is an explanatory diagram with respect to a fundamental concept of an inference processing method as the embodiment.Fig. 6 is an explanatory diagram with respect to the inference processing method as the first embodiment.Fig. 7 is a diagram illustrating visualized information of analysis information.Fig. 8 is a flow chart showing an example of processing procedures for realizing the inference processing method as the first embodiment.Fig. 9 is an explanatory diagram with respect to a modification of the first embodiment.Fig. 10 is an explanatory diagram of an inference processing method as a second embodiment.Fig. 11 is an explanatory diagram with respect to a specific example of the inference processing method as the second embodiment.Fig. 12 is an explanatory diagram with respect to a configuration example of an information processing apparatus as the second embodiment.Fig. 13 is an explanatory diagram with respect to a modification of the second embodiment.Fig. 14 is an explanatory diagram with respect to a first alternative example in the second embodiment.Fig. 15 is a diagram for explaining a configuration example of an information processing apparatus as the first alternative example in the second embodiment.Fig. 16 is an explanatory diagram with respect to a second alternative example in the second embodiment.Fig. 17 is a diagram for explaining a configuration example of an information processing apparatus as the second alternative example in the second embodiment.Fig. 18 is an explanatory diagram of an information processing apparatus as a third embodiment.Fig. 19 is a flow chart showing an example of processing procedures for realizing an inference processing method as the third embodiment.Fig. 20 is an explanatory diagram of an information processing apparatus as a fourth embodiment.

[0011] Hereinafter, embodiments according to the present technique will be described in the following order with reference to the accompanying drawings. < 1.　　　First Embodiment > (1-1.　　　Outline of System Configuration) (1-2.　　　Configuration Example of Sensing Apparatus) (1-3.　　　Example of Hardware Configuration of Information Processing Apparatus) (1-4.　　　Inference Processing Method as First Embodiment) (1-3.　　　Processing Procedures) (1-4.　　　Modification of First Embodiment) < 2.　　　Second Embodiment > (2-1.　　　Inference Processing Method as Second Embodiment) (2-2.　　　Alternative Examples of Second Embodiment) (2-2-1.　　　First Alternative Example) (2-2-2.　　　Second Alternative Example) < 3.　　　Third Embodiment > < 4.　　　Fourth Embodiment > < 5.　　　Modifications > < 6.　　　Summary of Embodiments > < 7.　　　Present Technique >

[0012] < 1.　　　First Embodiment > (1-1.　　　Outline of System Configuration) Fig. 1 is a block diagram showing a schematic configuration example of an inference system as a first embodiment according to the present technique. As illustrated, the inference system according to the first embodiment includes an information processing apparatus 1, a sensing apparatus 3, a user terminal 4, and a database 5. The information processing apparatus 1, the user terminal 4, and the database 5 are respectively configured as a computer apparatus equipped with a microcomputer including a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). In the present example, the information processing apparatus 1 is configured to enable data communication via a network 2 being the Internet or the like to be performed among the sensing apparatus 3, the user terminal 4, and the database 5.

[0013] The sensing apparatus 3 is an apparatus configured to include a sensor that senses a physical phenomenon (in other words, converts a physical quantity into an electric signal).　In the present example, an imaging apparatus is used as the sensing apparatus 3.

[0014] In the present specification, the term “imaging” shall broadly mean obtaining image data of a captured subject.　In this case, “image data” is a generic term for data made up of a plurality of pieces of pixel data being a concept that broadly includes not only data indicating an intensity of light received from a subject but also a distance to the subject, polarization information of the subject, temperature information, and the like.　In other words, “image data” (captured image data) obtained by “imaging” includes data as a grayscale image that indicates information on an intensity of an amount of light received by each pixel, data as a distance image that indicates a distance to the subject from each pixel, data as a polarized image that indicates polarization information of incident light to each pixel, data as a thermal image that indicates temperature information of each pixel, and the like.　In addition, “image data” also includes data as an event image obtained by an EVS (Event-based Vision Sensor) which includes an event sensor in which a plurality of event detection pixels that detect a change in the amount of received light as an event are two-dimensionally arrayed.　The data as an event image can also be described as image data indicating the presence or absence of an occurrence of an event for each pixel and can be expressed as image data capturing a motion of the subject.

[0015] In the inference system, the user terminal 4 is a computer apparatus that is expected to be used by a user being a recipient of a service using the inference system.　In addition, the information processing apparatus 1 is a computer apparatus that is expected to be used by a provider of the service.

[0016] The database 5 is configured to include a storage device for storing data and is used to store data obtained by the sensing apparatus 3.

[0017] The inference system according to the present example is configured as a system for providing a service that involves performing inference processing using an AI (Artificial Intelligence) model on sensing data from the sensing apparatus 3 (in the present example, captured image data) and, based on an inference result, generating analysis information indicating an analysis result of a target of sensing and presenting the analysis information to the user.

[0018] In this case, when a service of presenting the user with analysis information on the target of sensing based on an inference result as described above is assumed, conceivable applications for the sensing apparatus 3 include applications as various types of surveillance cameras.　Examples of such applications include an indoor surveillance camera at a store, an office, a residence, or the like, a surveillance camera (including a traffic surveillance camera) for monitoring outdoors at a parking lot or on the street, a surveillance camera of a production line in FA (Factory Automation) or IA (Industrial Automation), and a surveillance camera that monitors inside or outside a vehicle.

[0019] For example, in the application as a surveillance camera at a store, conceivably, a plurality of sensing apparatuses 3 are respectively arranged at predetermined positions inside the store in order to enable the user to check demographics (gender, age group, and the like), actions (flow lines) in the store, and the like of customers.　In this case, information on the demographics of the customers, information on the flow lines of the customers in the store, information on a state of congestion at checkout registers (for example, information on a wait time at the checkout registers), or the like is conceivably generated as the analysis information described above. Alternatively, in the application as a traffic surveillance camera, conceivably, each sensing apparatus 3 is arranged at each roadside position in order to enable the user to recognize a license plate (vehicle number), a body color, a make and model, or the like of passing vehicles and, in this case, information on the license plate, the body color, the make and model, or the like of the vehicle is conceivably generated as the analysis information described above.

[0020] In addition, when a traffic surveillance camera is used at a parking lot, conceivably, the camera is arranged so that each parked vehicle can be monitored to monitor whether or not a suspicious person acting suspiciously is present around each vehicle and, when there is a suspicious person, a notification of the presence of the suspicious person or attributes (gender, age group, clothing, and the like) of the suspicious person is issued. Furthermore, also conceivably, available spaces in town or in a parking lot may be monitored to notify a user of spaces where a vehicle can be parked.

[0021] Although Fig. 1 shows an example in which an inference system includes a plurality of sensing apparatuses 3, the number of sensing apparatuses 3 need only be at least one. In addition, although Fig. 1 only shows one user terminal 4 in the inference system, there may be a plurality of user terminals 4.　In other words, it is assumed that a plurality of users may be served by the inference system.

[0022] (1-2.　　　Configuration Example of Sensing Apparatus) Fig. 2 is a block diagram showing a configuration example of the sensing apparatus 3. As illustrated, the sensing apparatus 3 includes an image sensor 30, an imaging optical system 31, an optical system drive unit 32, a camera control unit 33, a memory unit 34, a communicating unit 35, and a sensor unit 36.　The image sensor 30, the camera control unit 33, the memory unit 34, the communicating unit 35, and the sensor unit 36 are connected via a bus 37 and are capable of performing data communication with each other.

[0023] In the present example, the image sensor 30 is configured as a grayscale image sensor that obtains the grayscale image described earlier.　In other words, the sensing apparatus 3 according to the present example is configured as a camera apparatus that obtains a grayscale image as a captured image.

[0024] The imaging optical system 31 includes a lens such as a cover lens, a zooming lens, a focusing lens and an aperture (iris) mechanism.　Due to the imaging optical system 31, light (incident light) from the subject is guided and collected on a light-receiving surface of the image sensor 30.

[0025] The optical system drive unit 32 is a comprehensive term for drive units of the zooming lens, the focusing lens, and the aperture mechanism included in the imaging optical system 31.　Specifically, the optical system drive unit 32 includes actuators and drive circuits of the actuators for respectively driving the zooming lens, the focusing lens, and the aperture mechanism.

[0026] The camera control unit 33 is constituted of, for example, a microcomputer including a CPU, a ROM, and a RAM and controls the sensing apparatus 3 as a whole as the CPU executes various types of processing according to a program stored in the ROM or a program loaded to the RAM.

[0027] In addition, the camera control unit 33 issues drive instructions for the zooming lens, the focusing lens, the aperture mechanism, and the like to the optical system drive unit 32.　The optical system drive unit 32 is to move the focusing lens or the zooming lens, open or close aperture blades of the aperture mechanism, and the like in accordance with the drive instructions.

[0028] Furthermore, the camera control unit 33 performs control of reading and writing various types of data from and to the memory unit 34. The memory unit 34 is, for example, a non-volatile storage device such as an HDD (Hard Disc Drive) or a flash memory apparatus and is used to store data used when the camera control unit 33 executes various types of processing.　In addition, the memory unit 34 can also be used as a storage (recording destination) of image data output from the image sensor 30.

[0029] The camera control unit 33 performs various types of data communication with external apparatuses via the communicating unit 35.　The communicating unit 35 according to the present example is configured to be capable of communicating via the network 2 shown in Fig. 1 and performing data communication with external apparatuses connected to the network 2 and, in particular, at least between the database 5 and the information processing apparatus 1 in the present example.

[0030] The sensor unit 36 comprehensively represents sensors other than the image sensor 30 included in the sensing apparatus 3.　Examples of sensors included in the sensor unit 36 include a GNSS (Global Navigation Satellite System) sensor or an altitude sensor for detecting a position or an altitude of the sensing apparatus 3, a temperature sensor for detecting ambient temperature, and a motion sensor such as an acceleration sensor or an angular velocity sensor for detecting motion of the sensing apparatus 3.　Further examples include an illuminance sensor that detects illuminance of outside light with respect to the sensing apparatus 3 and a sensor as a microphone for collecting sound around the sensing apparatus 3.

[0031] The image sensor 30 is configured as, for example, a solid-state imaging element such as a CCD (Charge Coupled Device)-type imaging element or a CMOS (Complementary Metal Oxide Semiconductor)-type imaging element and, as illustrated, includes an imaging unit 41, an image signal processing unit 42, an in-sensor control unit 43, an AI processing unit 44, a memory unit 45, a computer vision processing unit 46, and a communication interface (I / F) 47 which are capable of performing data communication with each other via a bus 48.

[0032] The imaging unit 41 includes a pixel array portion in which pixels having photoelectric conversion elements such as photodiodes are arrayed two-dimensionally and a readout circuit that reads out electrical signals obtained by photoelectric conversion from each pixel in the pixel array portion. The readout circuit executes, for example, CDS (Correlated Double Sampling) processing, AGC (Automatic Gain Control) processing, and A / D (Analog / Digital) conversion processing with respect to electric signals obtained by photoelectric conversion.

[0033] The image signal processing unit 42 performs preprocessing, synchronization processing, YC generation processing, resolution conversion processing, codec preprocessing, and the like with respect to captured image signals as digital data after A / D conversion processing.　In preprocessing, clamping in which black levels of R (red), G (green), and B (blue) are clamped to a predetermined level, correction processing between color channels of R, G, and B, or the like is performed with respect to a captured image signal.　In addition, in preprocessing, adjustment processing related to brightness such as gamma correction processing and adjustment processing related to color such as white balance adjustment processing can also be performed.　In synchronization processing, color separation processing in order to make image data with respect to each pixel include all of R, G, and B color components is performed.　For example, in a case of an imaging element using a Bayer-pattern color filter, demosaicking is performed as color separation processing.　In YC generation processing, a luminance (Y) signal and a color (C) signal are generated (separated) from R, G, and B image data.　In resolution conversion processing, resolution conversion processing is executed with respect to image data having been subjected to various types of signal processing. In codec processing, for example, coding processing for recording or communication and file generation are performed with respect to image data having been subjected to the various types of processing described above.　In codec processing, file generation can be performed in formats such as MPEG-2 (MPEG: Moving Picture Experts Group) and H.264 as file formats of moving images.　Furthermore, conceivably, file generation can also be performed in formats such as JPEG (Joint Photographic Experts Group), TIFF (Tagged Image File Format), and GIF (Graphics Interchange Format) as still image formats.

[0034] The in-sensor control unit 43 is constituted of, for example, a microcomputer configured to include a CPU, a ROM, and a RAM and comprehensively controls operations of the image sensor 30.　For example, the in-sensor control unit 43 controls execution of an imaging operation by issuing an instruction to the imaging unit 41.　In addition, the in-sensor control unit 43 also controls execution of processing with respect to the image signal processing unit 42.

[0035] The AI processing unit 44 is constituted of, for example, a programmable arithmetic processing unit such as a CPU, an FPGA (Field Programmable Gate Array), or a DSP (Digital Signal Processor) and performs processing (AI processing) using an AI model with respect to a captured image. The AI processing unit 44 according to the present example is configured to be capable of switching among AI models to be used. Details of the AI processing performed by the AI processing unit 44 will be described later.

[0036] The memory unit 45 is constituted of a volatile memory and is used to hold (temporarily store) data necessary for the AI processing unit 44 to perform AI processing.　Specifically, the memory unit 45 is used to store AI model data that is parameter data necessary for the AI processing unit 44 to construct an AI model.　When the AI model includes a neural network such as a CNN (Convolutional Neural Network) or the like, a parameter indicating a structure of the neural network or a parameter as a filter coefficient used in convolution processing or the like corresponds to the AI model data. In the present example, the memory unit 45 is also used to hold captured image data to be input data to the AI processing unit 44.

[0037] The computer vision processing unit 46 performs rule-based image processing as image processing with respect to captured image data.　Examples of the rule-based image processing in this case include super resolution processing.

[0038] The communication interface 47 is an interface that performs communication with each unit connected via the bus 37 such as the camera control unit 33, the memory unit 34, and the like outside of the image sensor 30.　For example, the communication interface 47 performs communication for acquiring AI model data and the like for realizing an AI model in the AI processing unit 44 from the outside based on control by the in-sensor control unit 43. In addition, information on a result of the AI processing by the AI processing unit 44 and the like can be output to the outside of the image sensor 30 via the communication interface 47.

[0039] (1-3.　　　Example of Hardware Configuration of Information Processing Apparatus) Fig. 3 is a block diagram showing an example of a hardware configuration of the information processing apparatus 1. Note that the computer apparatuses as the user terminal 4 and the database 5 shown in Fig. 1 may also conceivably adopt a similar hardware configuration to that shown in Fig. 3 (excluding an AI processing unit 14).

[0040] As illustrated, the information processing apparatus 1 includes a CPU 11.　The CPU 11 executes various types of processing in accordance with a program stored in a ROM 12 or a program loaded to a RAM 13 from the storage unit 19.　The RAM 13 also stores data required by the CPU 11 to execute the various types of processing when appropriate.

[0041] In a similar manner to the AI processing unit 44 described earlier, the AI processing unit 14 is constituted of, for example, a programmable arithmetic processing unit such as a CPU, an FPGA, or a DSP and performs processing (AI processing) using an AI model with respect to a captured image. In a similar manner to the AI processing unit 44, the AI processing unit 14 is also configured to be capable of switching among AI models to be used. Details of the AI processing performed by the AI processing unit 14 will also be described later.

[0042] The CPU 11, the ROM 12, the RAM 13, and the AI processing unit 14 are connected to each other via a bus 23.　An input / output interface (I / F) 15 is also connected to the bus 23.

[0043] An input unit 16 constituted of an operator or an operating device is connected to the input / output interface 15.　For example, various operators and operating devices such as a keyboard, a mouse, keys, a dial, a touch panel, a touch pad, and a remote controller are envisioned as the input unit 16. A user operation is detected by the input unit 16 and a signal corresponding to an input operation is interpreted by the CPU 11.

[0044] A display unit 17 constituted of an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence) panel, or the like and a sound output unit 18 constituted of a speaker or the like are integrally or separately connected to the input / output interface 15. The display unit 17 is used to display various types of information and is constituted of, for example, a display device provided in a housing of a computer apparatus or a separate display device connected to a computer apparatus.

[0045] The display unit 17 executes display of an image to be subjected to various types of image processing, a moving image to be processed, and the like on a display screen based on instructions of the CPU 11.　In addition, the display unit 17 displays various operation menus, icons, messages, and the like or, in other words, a GUI (Graphical User Interface) based on instructions of the CPU 11.

[0046] A storage unit 19 constituted of an HDD, a solid-state memory, or the like and a communicating unit 20 constituted of a modem or the like may be connected to the input / output interface 15.

[0047] The communicating unit 20 performs communication processing via through transmission channels such as the Internet, wired / wireless communication with various devices, bus communication, and the like.

[0048] When necessary, a drive 21 is connected to the input / output interface 15 and, when appropriate, a removable recording medium 22 that is a magnetic disk, an optical disk, a magneto optical disk, a semiconductor memory, or the like is mounted to the drive 21.

[0049] The drive 21 enables a data file and the like of a program used for each type of processing to be read from the removable recording medium 22.　The read data file is stored in the storage unit 19 or an image or sound contained in the data file is output from the display unit 17 or the sound output unit 18.　In addition, a computer program or the like read from the removable recording medium 22 is installed in the storage unit 19 when necessary.

[0050] In the computer apparatus with the hardware configuration described above, for example, software for processing according to the present embodiment can be installed via network communication by the communicating unit 20 or via the removable recording medium 22.　Alternatively, the software may be stored in the ROM 12, the storage unit 19, or the like in advance. Due to the CPU 11 performing processing operations based on various programs, information processing and communication processing necessary as the information processing apparatus 1 are executed.

[0051] Note that the information processing apparatus 1 is not limited to being solely constituted of a computer apparatus such as that shown in Fig. 3 and may be constituted of a system made up of a plurality of computer apparatuses.　The plurality of computer apparatuses may be made into a system by a LAN (Local Area Network) or the like or arranged at remote locations by a VPN (Virtual Private Network) using the Internet or the like.　The plurality of computer apparatuses may include computer apparatuses as a group of servers (cloud) that can be used by a cloud computing service.

[0052] (1-4.　　　Inference Processing Method as First Embodiment) First, a fundamental concept of an inference processing method as the embodiment will be described with reference to Figs. 4 and 5. Fig. 4 is an explanatory diagram with respect to functional partitioning of an AI model that is premised in the embodiment, and Fig. 4A schematically shows a network structure of an AI model using a DNN (Deep Neural Network). Generally, in an AI model, when an input layer side in a network is considered a front side and an output layer side is considered a back side, a portion of a stage preceding a given intermediate layer is called a backbone part and functions as a portion that extracts a feature amount of a target of sensing from input data. In addition, a portion of a stage following the backbone part in the network functions as a portion that performs processing of obtaining an inference result by using the extracted feature amount as input data and, in this case, the portion will be called a head part.

[0053] In the present embodiment, an AI model that performs a series of processing from extracting a feature amount to obtaining an inference result will be divided and handled as a backbone part being a preceding-stage portion of a network and a head part being a following-stage portion of a network as shown in Fig. 4B. Hereinafter, as shown in the drawings, an AI model that performs processing as the backbone part and an AI model that performs processing as the head part will be denoted as a “backbone model BB” and a “head model HD,” respectively.

[0054] Note that the backbone part and the head part may be respectively constituted of a plurality of models.　For example, the backbone model BB may include a neck model NC that performs feature amount extraction and feature amount integration and the head model HD may use a feature amount output from the neck model NC as input data.

[0055] In addition, in the following description, data indicating a feature amount extracted from input data by the backbone model BB will be denoted as “feature amount data”.

[0056] Fig. 5 is an explanatory diagram with respect to a fundamental concept of an inference processing method as the embodiment. As illustrated, the embodiment adopts a method in which the sensing apparatus 3 performs feature amount extraction using the backbone model BB and the information processing apparatus 1 executes inference processing using feature amount data extracted by the backbone model BB as input data using the head model HD.

[0057] In this case, conceivably, the backbone model BB and the head model HD are conceivably obtained by machine-learning an AI model including an entire network from an input layer to an output layer as shown in Fig. 4A described earlier and dividing the machine-trained AI model. In the present example, the AI processing is AI processing on image data and, for example, object detection processing or object recognition processing is conceivably performed.　An AI model for realizing a predetermined inference task such as object detection and object recognition is machine-learned and a trained AI model is divided at a predetermined intermediate layer position to obtain the backbone model BB and the head model HD.

[0058] Depending on a type of the inference task, there may be cases where the backbone model BB can be shared between different head models HD.　In such a case, for example, learning of the backbone model BB and learning of the head model HD may conceivably be separately performed.

[0059] Based on the assumptions described above, an inference processing method as the first embodiment will be described with reference to Fig. 6. First, in the present embodiment, feature amount data extracted by the sensing apparatus 3 using the backbone model BB is temporarily stored in the database 5.　As illustrated, in the present example, feature amount data obtained by each of a plurality of sensing apparatuses 3 is stored in the database 5. In the present example, the sensing apparatus 3 as an imaging apparatus performs feature amount extraction using the backbone model BB for each frame of a captured image and feature amount data of each frame is stored from each sensing apparatus 3 to the database.

[0060] In addition, in the present example, the sensing apparatus 3 adds ancillary information to feature amount data.　The ancillary information is to be added to the feature amount data of each frame and examples thereof include a time stamp (information representing a time of day of sensing: in this case, a time of day also includes a concept of date) and information on a location of sensing.　The information on a location of sensing can be acquired from a GNSS sensor included in the sensor unit 36 shown in Fig. 2. In addition, the ancillary information may include detected information by the various types of sensors included in the sensor unit 36 such as the illuminance sensor and the temperature sensor described earlier and information indicating weather at the time of sensing (for example, acquired by the sensing apparatus 3 from the Internet). Furthermore, for example, the ancillary information may conceivably include parameter information of signal processing performed on a captured image such as parameter information (for example, parameters of gamma correction or white balance adjustment) of image signal processing performed by the image signal processing unit 42 and drive information (accumulated time) of the sensors.

[0061] In the sensing apparatus 3, for example, the camera control unit 33 adds the ancillary information described above to feature amount data obtained by the image sensor 30, transmits the feature amount data to which the ancillary information has been added to the database 5 via the communicating unit 35, and causes the database 5 to store the feature amount data with the ancillary information.

[0062] Hereinafter, a set of feature amount data and ancillary information stored in the database 5 from each sensing apparatus 3 as described above will be denoted as “accumulated information 5a”.

[0063] The information processing apparatus 1 performs inference processing using the head model HD with respect to feature amount data stored as the accumulated information 5a in the database 5 as described above. Specifically, the information processing apparatus 1 includes: an input unit F1 that inputs feature amount data from the database 5; a selecting unit F2 that selects the head model HD based on input information of the user from a plurality of candidates of the head model HD; and a control unit F3 that performs control so that inference processing using the feature amount data input by the input unit F1 as input data is performed by the AI processing unit 14 using the head model HD selected by the selecting unit F2. In the present example, the input unit F1, the selecting unit F2, and the control unit F3 are functional units that are realized by software processing by the CPU 11 in the information processing apparatus 1.

[0064] The information processing apparatus 1 according to the present example includes a model storage unit F4.　The model storage unit F4 stores a plurality of head models HD to be candidates of selection by the selecting unit F2. For example, the model storage unit F4 is a functional unit that is realized by a storage device included in the information processing apparatus 1 such as the storage unit 19.

[0065] In the present example, while input information of the user with respect to the selecting unit F2 is provided by the user terminal 4 as illustrated, information indicating an object task to be realized using inference processing is input as the input information of the user in the present example and the selecting unit F2 selects the head model HD capable of executing an inference task specified from the object task. In the present example, the input information of the user is information representing an object task in a language format.　Specifically, the input information of the user is input in a text data format. However, user input information in a language format is not limited to information based on text data.　For example, information in a format of voice input data using a microphone is conceivable.　In this case, input voice data may be converted into text data to be used for selection by the selecting unit F2.

[0066] In the present example, input information of the user indicating an object task is expected to be input according to a text such as “I want to know the amount of human traffic at XYZ Intersection today” or “I want to know the number of customers in Store A yesterday morning”.　In other words, input information is expected to include designation information that designates a time or a location.

[0067] Correspondingly, the selecting unit F2 according to the present example performs processing of selecting feature amount data corresponding to the time or the location designated by the designation information described above from the feature amount data stored in the database 5 as the feature amount data to be input by the input unit F1. Accordingly, when feature amount data with respect to sensing data at various times of day and locations are accumulated in the database 5, inference processing can be performed on sensing data at a time or a location desired by the user.

[0068] In addition, in the present example, the selecting unit F2 selects the head model HD and feature amount data based on the input information using an LLM (large language model).　In other words, the head model HD and the feature amount data necessary for realizing an object task are selected by performing a retrieval of the head model HD and the feature amount data using an LLM, with a language-format data indicating the object task input by the user as a prompt.

[0069] In order to accommodate such retrievals using an LLM, in the model storage unit F4, tag information for retrieval is attached to each head model HD.　Specifically, as the tag information, information (hereinafter, denoted as “inference task identification information”) indicating at least a type of an inference task of inference processing to be executed by the head model HD is attached.

[0070] For example, with respect to the input of the object task of “I want to know the amount of human traffic at XYZ Intersection today” described above, the selecting unit F2 selects the head model HD to which inference task identification information indicating object detection processing with respect to “human” has been added from the model storage unit F4 and selects feature amount data to which location information corresponding to “XYZ Intersection” and a time stamp corresponding to “today” have been added from the database 5.

[0071] Using an LLM in the selection processing by the selecting unit F2 enables a retrieval of the head model HD or the feature amount data matching task conditions designated by the user in a language format to be performed with high accuracy. Therefore, the probability that the object task desired by the user is properly executed can be increased.

[0072] In addition, the information processing apparatus 1 includes functions as an analyzing unit F5 and a presentation processing unit F6.　In the present example, the functions of the analyzing unit F5 and the presentation processing unit F6 are also functions to be realized by software processing by the CPU 11.

[0073] The analyzing unit F5 inputs inference result information (in other words, information indicating an inference result corresponding to an object task designated by the user) by the AI processing unit 14, performs analysis processing using the input inference result information, and generates analysis information of the target of sensing.　The analyzing unit F5 performs analysis processing according to the object task designated by the user by input information.　For example, when the designated object task is “I want to know the amount of human traffic at XYZ Intersection today”, the analyzing unit F5 generates information indicating an amount of human traffic at a relevant location and a relevant time as analysis information by performing processing such as counting the number of people using information on a detection result of people obtained by the AI processing unit 14.

[0074] The presentation processing unit F6 performs processing of presenting the user with the analysis information obtained by the analyzing unit F5 as visualized information. Fig. 7 illustrates visualized information of analysis information.　Specifically, visualized information is information that visualizes and shows, for each predetermined time slot, an amount of human traffic at a given location and during a given period. For example, the presentation processing unit F6 performs processing of generating image information indicating an analysis result such as that shown in Fig. 7 as visualized information of the analysis information and presenting the user with the generated visualized information via the user terminal 4.　In other words, image information as the visualized information is transmitted to the user terminal 4 to be displayed on the display screen of the user terminal 4.

[0075] While an example of the presentation processing unit F6 generating the visualized information of analysis information has been described above, a configuration in which the analyzing unit F5 performs processing up to the generation of visualized information is also conceivable.

[0076] In addition, while an example in which candidates of the head model HD are stored in the model storage unit F4 in the information processing apparatus 1 has been described above, the candidates of the head model HD may be stored in a storage apparatus outside of the information processing apparatus 1 such as the database 5.

[0077] (1-3.　　　Processing Procedures) Fig. 8 is a flow chart showing an example of processing procedures for realizing the inference processing method as the first embodiment described above. In the present example, the processing shown in Fig. 8 is executed by the CPU 11 in the information processing apparatus 1 based on, for example, a program stored in the ROM or the storage unit 19.

[0078] In this case, it is assumed that the feature amount data and ancillary information thereof extracted by the backbone model BB have already been accumulated in the database 5 from the sensing apparatus 3 by the time the processing shown in Fig. 8 is executed.　The backbone model BB required to realize the object task designated by the input information of the user is deployed with respect to the sensing apparatus 3.　Specifically, the backbone model BB is selected by the information processing apparatus 1 (CPU 11) based on input information of the user from a plurality of candidates and the selected model is deployed to the target sensing apparatus 3. For the sake of confirmation, “deploy” means to transmit an AI model to a target apparatus and cause the target apparatus to install the AI model.　Specifically, transmission of AI model data described earlier is performed and a setting instruction of an AI model based on the AI model data is issued.

[0079] In Fig. 8, in step S101, the CPU 11 performs retrieval processing of the head model HD and the feature amount data based on user input information.　Specifically, in the present example, retrieval processing of the head model HD and the feature amount data using an LLM is performed, with information indicating an object task input by the user in a language format as a prompt.

[0080] In step S102 following step S101, the CPU 11 sets the head model HD selected based on a retrieval result to the AI processing unit 14.　Specifically, a relevant AI model is set to the AI processing unit 14 according to the AI model data for realizing the head model HD.

[0081] In step S103 following step S102, the CPU 11 inputs the feature amount data selected based on a retrieval result.　In other words, based on the retrieval result of step S101, relevant feature amount data is selected from accumulated information 5a of the database 5 and the selected feature amount data is input.

[0082] In step S104 following step S103, the CPU 11 causes the AI processing unit 14 to execute inference processing using the input feature amount data as input data.

[0083] In step S105 following step S104, the CPU 11 executes analysis processing based on an inference result and, furthermore, in subsequent step S106, the CPU 11 performs processing of presenting the user with visualized information of analysis information. Accordingly, information indicating a result of the object task designated by the user can be presented to the user by visualized information.

[0084] The CPU 11 ends the series of processing shown in Fig. 8 in accordance with executing the processing of step S106.

[0085] (1-4.　　　Modification of First Embodiment] While the description given above assumes that the number of head models HD required to realize the object task designated by the user is one, there may be cases where a plurality of head models HD are required to realize the object task.　For example, in the case of an object task of “I want to know the amount of human traffic and vehicle traffic at XYZ Intersection today”, as illustrated in Fig. 9, the information processing apparatus 1 is to perform, using the head model HD (“HDh” in the drawing) for human detection and the head model HD (“HDc” in the drawing) for vehicle detection, inference processing using common feature amount data (the feature amount data selected by the selecting unit F2) as input data.

[0086] Therefore, when the number of head models HD required to realize the object task specified from input information of the user is a plurality of head models HD, the selecting unit F2 selects respective head models HD (in other words, a plurality of head models HD with different inference tasks) from the model storage unit F4 and the control unit F3 causes each head model HD selected by the selecting unit F2 to execute inference processing using the same piece of feature amount data input by the input unit F1 as input data.

[0087] Accordingly, inference processing of different tasks can be simultaneously performed with respect to a same target of sensing. In addition, since a single backbone model BB need only be used when performing inference processing of different tasks and the sensing apparatus 3 need no longer perform processing of feature amount extraction using the backbone model BB for each task, a reduction in processing load on the sensing apparatus 3 can be achieved.

[0088] < 2.　　　Second Embodiment > (2-1.　　　Inference Processing Method as Second Embodiment) Next, a second embodiment will be described. As shown in Fig. 10, the second embodiment deploys the head model HD to the sensing apparatus 3 and causes the sensing apparatus 3 to execute inference processing of a predetermined task. The information processing apparatus according to the second embodiment that deploys the head model HD to the sensing apparatus 3 will be denoted as an information processing apparatus 1A.　A hardware configuration of the information processing apparatus 1A may be similar to that of the information processing apparatus 1 and descriptions thereof will not be repeated.

[0089] In the following description, portions that are similar to those already described will be denoted by same reference signs and descriptions thereof will not be repeated.

[0090] For example, the second embodiment is intended to be capable of accommodating a case where an object task designated by input information of the user requires inference processing on data during sensing instead of data accumulated in the database 5.　Conceivably, examples of the object task in this case include “I want to know the current amount of human traffic at XYZ Intersection”.

[0091] In this case, the selecting unit F2 in the information processing apparatus 1A selects the relevant sensing apparatus 3 and the relevant head model HD based on the object task designated by the user.　In addition, the selected head model HD is deployed to the selected sensing apparatus 3 and the sensing apparatus is caused to execute inference processing. At this point, information of an inference result obtained by the sensing apparatus 3 may be temporarily stored in the database 5 and then transmitted to the information processing apparatus 1A or transmitted from the sensing apparatus 3 to the information processing apparatus 1A without involving the database 5. In this case, since analysis processing and presentation processing based on the inference result information are similar to the processing by the analyzing unit F5 and the presentation processing unit F6 described in the first embodiment, descriptions thereof will not be repeated.

[0092] In addition, in the second embodiment, when the selecting unit F2 selects a plurality of head models HD when the object task designated by the input information of the user requires a plurality of types of inference processing on data during sensing, as shown in Fig. 11, the head models HD (HD1 and HD2 in the drawing) are deployed to the sensing apparatus 3 and an instruction is issued to the sensing apparatus 3 to switch the head model HD to be used between the deployed head models HD.

[0093] Accordingly, since a single sensing apparatus 3 can be caused to perform inference processing with respect to the data during sensing by a plurality of head models HD in a time-shared manner and there is no longer a need to deploy each head model HD to the plurality of sensing apparatuses 3 and cause each head model HD to execute processing, a reduction in the number of sensing apparatuses can be achieved.

[0094] As shown in Fig. 12, the information processing apparatus 1A according to the second embodiment differs from the information processing apparatus 1 in that the information processing apparatus 1A includes a CPU 11A including a control unit F3A in place of the control unit F3. The control unit F3A performs control so that the head model HD selected by the selecting unit F2 is deployed to the sensing apparatus 3. In particular, when the selecting unit F2 selects a plurality of head models HD based on the input information of the user, the control unit F3A performs control so that the plurality of head models HD are deployed to the target sensing apparatus 3 and issues an instruction to the sensing apparatus 3 to switch the head model HD to be used among the deployed head models HD.

[0095] Although a case where a plurality of head models HD are required to realize the object task designated by a single user has been exemplified above as an example of the selecting unit F2 selecting a plurality of head models HD, the selecting unit F2 may select a plurality of head models HD in order to realize an object task of each of a plurality of users requesting the execution of different object tasks based on input information from the plurality of users. When the control unit F3A performs the processing described above in accordance with such a case, since a single sensing apparatus 3 can be caused to execute inference processing for the object task of each user in a time-shared manner, there is no longer a need to prepare a sensing apparatus 3 for each user when realizing the respective object tasks of the plurality of users and a reduction in the number of sensing apparatuses can be achieved.

[0096] By deploying a plurality of head models HD to the sensing apparatus 3 as described above, a single sensing apparatus 3 can be caused to simultaneously execute inference processing with respect to a plurality of pieces of real-time sensing data.　In this case, a plurality of AI processing units 44 in order to simultaneously operate the plurality of head models HD are to be prepared in the sensing apparatus 3.　An application in this case is not limited to simultaneously executing a plurality of tasks and an application of increasing the number of classes that can be identified by increasing the number of head models HD when the number of identifiable classes by a single head model HD is not sufficient is also conceivable.

[0097] In addition, in the second embodiment, as shown in Fig. 13, a situation where the information processing apparatus 1A not only deploys the plurality of head models HD but also deploys a plurality of backbone models BB (BB1 and BB2 in the drawing) and issues a switching instruction of the backbone models BB is also conceivable. Accordingly, cases where the backbone model BB that can be accommodated by each head model HD differs can be accommodated.

[0098] In this case, switching between backbone models BB can include switching to the backbone model BB with a different number of layers. Accordingly, cases where the number of layers of the backbone model BB that can be accommodated by each head model HD differs can be accommodated.

[0099] For the sake of confirmation, when the user issues an instruction to execute inference processing on the feature amount data in the accumulated information 5a, the control unit F3A according to the second embodiment also performs control so that inference processing using the feature amount data input by the input unit F1 as input data is performed using the head model HD selected by the selecting unit F2 in a similar manner to the control unit F3 according to the first embodiment. Similarly, in the control units (control units F3B to F3E) according to the respective embodiments to be described hereinafter, when the user issues an instruction to execute inference processing on the feature amount data in the accumulated information 5a, the control units are to perform control so that inference processing using the feature amount data input by the input unit F1 as input data is performed using the head model HD selected by the selecting unit F2.

[0100] (2-2.　　　Alternative Examples of Second Embodiment) (2-2-1.　　　First Alternative Example) Information processing apparatuses as alternative examples of the second embodiment will now be described.　In this case, two examples, namely, a first alternative example and a second alternative example will be described and the information processing apparatus as the first alternative example will be denoted as an information processing apparatus 1B and the information processing apparatus as the second alternative example will be denoted as an information processing apparatus 1C.

[0101] As shown in Fig. 14, the information processing apparatus 1B as the first alternative example issues, with respect to the sensing apparatus 3 that is a deployment destination of the head model HD, an instruction to transfer the deployed head model HD to another sensing apparatus 3.

[0102] As shown in Fig. 15, the information processing apparatus 1B differs from the information processing apparatus 1 in that the information processing apparatus 1B includes a CPU 11B including the control unit F3A in place of the control unit F3. The control unit F3B performs control so that the head model HD selected by the selecting unit F2 is deployed to the target sensing apparatus 3 and issues an instruction to the sensing apparatus 3 that is the deployment destination to transfer the deployed head model HD to another sensing apparatus 3.

[0103] Accordingly, when performance of inference processing with the head model HD selected by the selecting unit F2 is requested in not only a single sensing apparatus 3 but also in another sensing apparatus 3, there is no longer a need to perform deployment processing of the head model HD to the other sensing apparatus 3.

[0104] The transfer function of the head model HD as the first alternative example described above can be suitably applied to tracking of a specific object such as a criminal or a stolen vehicle. In this case, the selecting unit F2 selects the head model HD to perform object detection processing on the specific object.　In addition, the control unit F3B issues the following instruction as the transfer instruction described above with respect to the sensing apparatus 3 that is the deployment destination.　Specifically, an instruction is issued indicating that a transfer of the deployed head model HD to another sensing apparatus 3 is to be performed under the condition that an inference result of a detection of the specific object by inference processing performed by the deployed head model HD is obtained.

[0105] Accordingly, in the sensing apparatus 3 to which the head model HD has been deployed, the head model HD for detecting the specific object that is a criminal or a stolen vehicle can be arranged to be transferred to another sensing apparatus 3 nearby in response to the detection of the specific object, thereby enabling the specific object to be tracked.

[0106] In this case, if a position of each sensing apparatus 3 is known, the sensing apparatus 3 that is the deployment destination can specify another sensing apparatus 3 positioned in a direction of movement of the specific object (by performing inference processing of a direction of movement of the detected specific object or the like) and transfer the head model HD to the specified sensing apparatus 3.

[0107] (2-2-2.　　　Second Alternative Example) The second alternative example issues an instruction to transmit feature amount data extracted by the backbone model BB to another sensing apparatus 3 instead of issuing a transfer instruction of the deployed head model HD. Specifically, as shown in Fig. 16, the information processing apparatus 1C as the second alternative example causes a plurality of head models HD selected based on input information of the user to be respectively deployed to different sensing apparatuses 3 and issues an instruction to one of the sensing apparatuses 3 that are deployment destinations to transmit the feature amount data extracted by the one sensing apparatus 3 to another sensing apparatus 3 that is a deployment destination.

[0108] Accordingly, when a plurality of types of inference processing are to be performed with respect to one piece of data during sensing, the inference processing can be realized as distributed processing using the plurality of sensing apparatuses 3.

[0109] As shown in Fig. 17, the information processing apparatus 1C differs from the information processing apparatus 1 in that the information processing apparatus 1C includes a CPU 11C including a control unit F3C in place of the control unit F3. The control unit F3C performs control so that the plurality of head models HD selected by the selecting unit F2 are respectively deployed to different sensing apparatuses 3. The selecting unit F2 in this case selects the plurality of head models HD in response to a case where an object task requiring execution of a plurality of different types of inference processing is designated with respect to data during sensing of a given single location by input information of a single user. In addition, the control unit F3C performs deployment control of the plurality of head models HD selected by the selecting unit F2 as described above and issues an instruction to one of the sensing apparatuses 3 that are deployment destinations to transmit the feature amount data extracted by the backbone model BB of the one sensing apparatus 3 to another sensing apparatus 3 that is a deployment destination.

[0110] < 3.　　　Third Embodiment > A third embodiment relates to relearning of a backbone model BB. An information processing apparatus 1D as the third embodiment will be described with reference to Fig. 18. The information processing apparatus 1D differs from the information processing apparatus 1 in that the information processing apparatus 1D includes a control unit F3D in place of the control unit F3 and also includes a relearning processing unit F7.

[0111] The control unit F3D performs execution control of relearning with respect to the backbone model BB used by the sensing apparatus 3 in accordance with the fulfillment of a predetermined trigger condition and performs control so that the backbone model BB obtained by the relearning (hereinafter, denoted as “backbone model BB’” as shown in the drawing) is deployed to the sensing apparatus 3. In the present example, the “predetermined trigger condition” as an execution condition for relearning is a condition based on an assessment of accuracy of the inference processing performed using the head model HD.　Specifically, the control unit F3D according to the present example causes the relearning processing unit F7 to perform relearning of the backbone model BB used by the sensing apparatus 3 on the condition that an error is detected in an inference result of the inference process performed using the head model HD with the feature amount data accumulated in the database 5 from the sensing apparatus 3 as input data. Whether or not there is an error in the inference result can be determined by the control unit F3D itself by performing processing of error detection.　For example, the processing of error detection can be realized as determination processing of whether or not an impossible inference result has been obtained.　Specifically, for example, when object detection processing on a specific object is premised on the detection of at least one specific object in a sensing environment, a determination of whether or not the number of detected specific objects is zero may conceivably be performed as the processing of error detection.　Alternatively, when a same area is being sensed by sensing apparatuses 3 at different angles and inference processing of a same task is being executed with respect to sensing data of each of the sensing apparatuses 3, a determination of whether or not one sensing apparatus 3 is unable to detect an object being detected by a plurality of other sensing apparatuses 3 may conceivably be performed as the processing of error detection.　Furthermore, there may be cases where an error is pointed out by a feedback of analysis information to the user and, in such a case, determination processing of whether or not an error has been pointed out may conceivably be performed as the processing of error detection.

[0112] While an example in which an assessment of the presence or absence of an error in an inference result is used as the “assessment of accuracy of the inference processing” has been described above, conceivably, an assessment based on a score (likelihood) with respect to the inference result may be used as the “assessment of accuracy of the inference processing”.　For example, when performing object recognition processing, conceivably, a determination of whether or not an average value of scores in a time direction with respect to a single given recognition target is equal to or lower than a threshold or a determination of whether or not an average value of scores of a plurality of recognition targets is equal to or lower than a threshold may be performed as a determination of whether or not the trigger condition has been fulfilled.

[0113] Fig. 19 is a flow chart showing an example of processing procedures for realizing the inference processing method as the third embodiment described above. The processing shown in Fig. 19 is executed by the CPU 11 (denoted as a “CPU 11D” for the purpose of distinction) in the information processing apparatus 1D based on a program stored in the ROM or the storage unit 19 in the information processing apparatus 1D.

[0114] First, in step S201, the CPU 11D stands by until a relearning condition of the backbone model BB is fulfilled.　Specifically, in the present example, a result of inference processing by the head model HD performed using feature amount data input from the target sensing apparatus 3 via the database 5 as input data is input to execute the error detection processing described above.

[0115] In step S201, when an error in the inference result is detected by the error detection processing and a determination that the relearning condition has been fulfilled is made, the CPU 11D advances to step S202 and issues a relearning execution instruction of the backbone model BB to the relearning processing unit F7.

[0116] Subsequently, in step S203 following step S202, the CPU 11D performs deployment processing of the relearned backbone model BB’ to a transmission source sensing apparatus.　In other words, control is performed so that the backbone model BB’ obtained by the relearning processing by the relearning processing unit F7 is deployed to the target sensing apparatus 3.　At this point, the CPU 11D also issues an instruction to the target sensing apparatus 3 to switch the backbone model to be used to the backbone model BB’.

[0117] The CPU 11D ends the series of processing shown in Fig. 19 in accordance with executing the processing of step S203.

[0118] While an example in which the relearning processing unit F7 is provided in the information processing apparatus 1D has been described above, the relearning processing may be performed by an apparatus other than the information processing apparatus 1D.

[0119] In addition, while a case where the “predetermined trigger condition” as an execution condition for relearning is a “condition based on an assessment of accuracy of the inference processing” has been exemplified above, the “predetermined trigger condition” is not limited thereto. For example, a situation is also conceivable where relearning of the backbone model BB is caused to be executed when a need to increase targets of object detection processing or object recognition processing arises due to a designation of a new object task by the user.

[0120] < 4.　　　Fourth Embodiment > A fourth embodiment relates to optimization of a backbone model BB in accordance with a sensing environment. An information processing apparatus 1E as the fourth embodiment will be described with reference to Fig. 20. As illustrated, the information processing apparatus 1E differs from the information processing apparatus 1 in that the information processing apparatus 1E includes a control unit F3E in place of the control unit F3. In addition, in the information processing apparatus 1E, the model storage unit F4 stores a plurality of candidates of the backbone model BB together with a plurality of candidates of the head model HD (not illustrated).

[0121] A hardware configuration of the information processing apparatus 1E may be similar to that of the information processing apparatus 1 and descriptions thereof will not be repeated.　The processing by the control unit F3E described below can be executed by software processing by the CPU 11 (denoted as a “CPU 11E” for the purpose of distinction) included in the information processing apparatus 1E.

[0122] In the model storage unit F4, each candidate of the backbone model BB is a model having been trained to be specific to an expected sensing environment.　Examples include a backbone model BB having been trained to be specific to a lighting environment with strong oblique irradiating light at sunrise or sunset and a backbone model BB having been trained to be specific to a lighting environment with an approximately uniform illuminance distribution and minimal oblique irradiating light.

[0123] The control unit F3E includes functions as an environment estimating unit F31, a model selecting unit F32, and a deployment processing unit F33. The environment estimating unit F31 performs processing of estimating a sensing environment with respect to the target sensing apparatus 3 based on ancillary information accumulated as the accumulated information 5a together with feature amount data in the database 5.

[0124] In this case, as exemplified earlier, the ancillary information includes a time stamp and information of a sensing location with respect to the feature amount data, a temperature or detected information of a motion sensor, information on the illuminance of outside light, information on an environmental sound detected by a microphone, and information on weather at the time of sensing. Such ancillary information can be described as environmental estimation basis information that can be used as a basis for estimation with respect to the sensing environment in the target sensing apparatus 3.

[0125] For example, based on ancillary information as described above, the environment estimating unit F31 estimates whether the sensing environment of the target sensing apparatus 3 is the environment with strong oblique light (oblique light environment) or the environment with an approximately uniform illuminance distribution (uniform illuminance environment) described above.　To this end, using the time stamp and the weather information in the ancillary information, the environment estimating unit F31 according to the present example estimates whether the sensing environment in a present time slot is the oblique light environment or the uniform illuminance environment described above.　For example, when the time slot is morning or evening and the weather is sunny, the oblique light environment can be estimated.　On the other hand, when the weather is cloudy or rainy even though the time slot is morning or evening, the uniform illuminance environment can be estimated instead of the oblique light environment. An improvement in environment estimation accuracy can be achieved by using a plurality of types of environmental estimation basis information.

[0126] The estimation processing of a sensing environment described above is merely an example and the estimation processing of a sensing environment is not limited thereto.　For example, the estimation of an environment can be an estimation from three or more types instead of an estimation from two types.　In addition, the information used in environment estimation described above is also merely an example and information according to estimation contents may be appropriately selected and used as the environmental estimation basis information.

[0127] The model selecting unit F32 selects the backbone model BB in accordance with the sensing environment estimated by the environment estimating unit F31 from among the plurality of candidates of the backbone model BB stored in the model storage unit F14.　In the drawing, the backbone model BB selected by the model selecting unit F32 is denoted as a “backbone model BBx”.

[0128] The deployment processing unit F33 performs control so that the backbone model BB selected by the model selecting unit F32 is deployed to the target sensing apparatus 3.　In other words, the deployment processing unit F33 causes the backbone model BBx to be deployed to the sensing apparatus 3 as a transmission source of the feature amount data to which the ancillary information used in environment estimation has been added.

[0129] The information processing apparatus 1E as the fourth embodiment described above enables the target sensing apparatus 3 to use the backbone model BB suitable for a sensing environment.　In other words, the target sensing apparatus 3 is enabled to use the backbone model BB capable of extracting an appropriate feature amount with respect to the sensing environment. Therefore, an improvement in inference performance can be achieved.

[0130] While an example of achieving the realization of inference processing suitable for an environment by preparing models having been trained to be specific to each environment as candidates of the backbone model BB has been described above, a wide range of needs of a user can be accommodated by utilizing a mechanism of deploying the backbone model BB selected according to an estimation environment as in the fourth environment to the sensing apparatus 3. For example, on the premise of a head model to be used being defined (by the user) for each environment such as using a first head model HD (for example: human detection) during morning time slots and using a second head model HD (vehicle detection) during night time slots, a method of selecting the backbone model BB corresponding to the head model HD to be used in an estimated environment from the candidates is conceivably adopted.

[0131] In the fourth embodiment, candidates of the backbone model BB are not limited to preparing models that use different parameters, and models with different depths (the numbers of intermediate layers) are also conceivably prepared.

[0132] In addition, in the fourth embodiment, the environmental estimation basis information is not limited to the information exemplified above and, for example, the environmental estimation basis information may include parameters (for example, a brightness adjustment value, a gamma correction value, a white balance adjustment value, and an adjustment value of saturation or hue) of image signal processing performed by the image signal processing unit 42, imaging parameters (for example, ISO sensitivity, a shutter speed (exposure time), and an aperture value) of the imaging unit 41.　In addition, the environmental estimation basis information may conceivably include information such as electricity consumption by the sensing apparatus 3.

[0133] Furthermore, in the fourth embodiment, the candidates of the backbone model BB may be stored in a storage apparatus which differs from that storing the candidates of the head model HD.　In addition, the candidates of the backbone model BB may be stored outside of the information processing apparatus 1E.

[0134] < 5.　　　Modifications > Embodiments are not limited to the specific examples described above and various configurations as modifications can be adopted. For example, while an imaging apparatus has been exemplified above as an example of the sensing apparatus 3, sensing according to the present technique is not limited to sensing of an image (sensing of subject light) and can include other sensing such as sensing of sound and sensing of acceleration or angular velocity.　For example, in the case of sensing of sound, examples of inference processing on sensing data include processing of conversion into text, processing of speaker separation (separation to determine which utterance was made by which speaker), and summarization processing of uttered contents. In the present technique, the sensing data need only be sensing data that can be made a target of inference processing by an AI model and the sensing need only be sensing capable of obtaining such sensing data.

[0135] < 6.　　　Summary of Embodiments > As described above, information processing apparatuses (information processing apparatuses 1, 1A, 1B, 1C, 1D, and 1E) as the embodiments include: an input unit (input unit F1) configured to input feature amount data from a database storing the feature amount data, the feature amount data having been obtained by a sensing apparatus including a sensor and configured to perform feature amount extraction on sensing data from the sensor using a backbone model being an AI model configured to perform feature amount extraction on input data; a selecting unit (selecting unit F2) configured to select, from a plurality of candidates of a head model being an AI model configured to perform inference processing using feature amount data as input data, the head model based on input information of a user; and control units (control units F3, F3A, F3B, F3C, F3D, and F3E) configured to perform control using the head model selected by the selecting unit so that inference processing using the feature amount data input by the input unit as input data is performed. A data amount of the feature amount data extracted by the backbone model from the sensing data is smaller than a data amount of the sensing data.　In addition, the feature amount data represents a feature amount of a target of sensing extracted from the sensing data and does not represent the target of sensing itself. Therefore, according to the configuration described above, when performing inference processing using an AI model based on data accumulated in a database from a sensing apparatus, privacy protection can be achieved while achieving reductions in a storage capacity of the database and an amount of communication data between the sensing apparatus and the database.

[0136] In addition, in the information processing apparatuses as the embodiments, information indicating an object task to be realized using the inference processing is input as input information of the user, and the selecting unit is configured to select the head model capable of executing an inference task specified from the object task. Accordingly, performance of inference processing for realizing the object task desired by the user can be ensured.

[0137] Furthermore, in the information processing apparatuses as the embodiments, the feature amount data is associated with ancillary information indicating at least any of a time of day of sensing and a sensing location of sensing data to be an extraction source of the feature amount data, the input information includes designation information that designates a time or a location, and the selecting unit is configured to select feature amount data corresponding to the time or the location designated by the designation information from the feature amount data stored in the database as feature amount data to be input by the input unit. Accordingly, when feature amount data with respect to sensing data at various times and locations is accumulated in the database, inference processing can be performed on sensing data at a time or a location desired by the user.

[0138] In addition, in the information processing apparatuses as the embodiments, the input information is information that expresses the object task in a language format, and the selecting unit is configured to select the head model and the feature amount data based on the input information using a large language model. Using a large-scale language model enables a retrieval of the head model or the feature amount data matching task conditions designated by the user in a language format to be performed with high accuracy. Therefore, the probability that the object task desired by the user is properly executed can be increased.

[0139] Furthermore, in the information processing apparatuses as the embodiments, the selecting unit is configured to select a plurality of head models with different inference tasks based on the input information, and the control unit is configured to cause each head model selected by the selecting unit to execute inference processing using a same piece of feature amount data input by the input unit as input data (see Fig. 9). Accordingly, inference processing of different tasks can be simultaneously performed with respect to a same target of sensing. In addition, since a single backbone model need only be used when performing inference processing of different tasks and the sensing apparatus need no longer perform processing of feature amount extraction processing using a backbone model for each task, a reduction in processing load on the sensing apparatus can be achieved.

[0140] In addition, the information processing apparatuses as the embodiments include a presentation processing unit (presentation processing unit F6) configured to perform processing of presenting the user with analysis information of a target of sensing generated based on inference result information by the head model as visualized information. Accordingly, information indicating a result of the object task designated by the user can be presented to the user by visualized information.

[0141] Furthermore, in the information processing apparatuses (information processing apparatuses 1A, 1B, and 1C) as some of the embodiments, the control units (control units F3A, F3B, and F3C) are configured to perform control so that the head model selected by the selecting unit is deployed to the sensing apparatus. Accordingly, when an object task designated by the user requires inference processing on data during sensing instead of data accumulated in the database, the sensing apparatus can be caused to execute inference processing on the data during sensing. Accordingly, a width of tasks that can be accommodated can be widened.

[0142] In addition, in the information processing apparatus (information processing apparatus 1A) as one of the embodiments, the selecting unit is configured to select a plurality of head models, and the control unit (control unit F3A) is configured to perform control so that the plurality of head models selected by the selecting unit are deployed to the sensing apparatus and to issue an instruction to the sensing apparatus to switch the head model to be used among the deployed head models. Accordingly, a case where types of inference processing using different head models are required to be executed by switching among the types of inference processing on a time axis as the inference processing to be executed by the sensing apparatus on data during sensing can be accommodated. For example, when the selecting unit selects a plurality of head models when a plurality of head models are required to realize an object task of a single user, since a single sensing apparatus can be caused to perform inference processing with respect to the data during sensing by a plurality of head models in a time-shared manner and there is no longer a need to deploy each head model to the plurality of sensing apparatuses and cause each head model to execute processing, a reduction in the number of sensing apparatuses can be achieved. Alternatively, when the selecting unit selects, based on input information from a plurality of users requesting the execution of different object tasks, a plurality of head models for realizing the object task of each of the users, since a single sensing apparatus can be caused to execute inference processing for the object task of each user in a time-shared manner, there is no longer a need to prepare a sensing apparatus for each user when realizing the respective object tasks of the plurality of users and a reduction in the number of sensing apparatuses can be achieved.

[0143] Furthermore, in the information processing apparatus (information processing apparatus 1B) as one of the embodiments, the control unit (control unit F3B) is configured to issue, with respect to the sensing apparatus to which the head model has been deployed, an instruction to transfer the deployed head model to another sensing apparatus. Accordingly, when performance of inference processing with the head model selected by the selecting unit is requested in not only a single sensing apparatus but also in another sensing apparatus, there is no longer a need to perform deployment processing of the head model to the other sensing apparatus. Therefore, a reduction in processing load in the information processing apparatus can be achieved.

[0144] In addition, in the information processing apparatus (information processing apparatus 1B) as one of the embodiments, the deployed head model is a head model configured to perform object detection processing on a specific object, and the control unit (control unit F3B) is configured to issue, as the instruction, an instruction indicating that a transfer of the deployed head model to the other sensing apparatus is to be performed under the condition that an inference result of a detection of the specific object by inference processing performed by the head model is obtained. Accordingly, in the sensing apparatus to which the head model has been deployed, for example, the head model for detecting the specific object that is a criminal or a stolen vehicle can be arranged to be transferred to another sensing apparatus nearby in response to the detection of the specific object, thereby enabling the specific object to be tracked.

[0145] Furthermore, in the information processing apparatus (information processing apparatus 1C) as one of the embodiments, the selecting unit is configured to select a plurality of head models, and the control unit (control unit F3C) is configured to perform control so that the plurality of head models selected by the selecting unit are respectively deployed to different sensing apparatuses and to issue an instruction to one of the sensing apparatuses being deployment destinations to transmit the feature amount data extracted by the backbone model of the one sensing apparatus to another of the sensing apparatuses being deployment destinations. Accordingly, when a plurality of types of inference processing are to be performed with respect to one piece of data during sensing, the inference processing can be realized as distributed processing using the plurality of sensing apparatuses.

[0146] In addition, in the information processing apparatus (information processing apparatus 1D) as one of the embodiments, the control unit (control unit F3D) is configured to perform execution control of relearning with respect to the backbone model used by the sensing apparatus in accordance with the fulfillment of a predetermined trigger condition and to perform control so that the backbone model obtained by the relearning is deployed to the sensing apparatus. Accordingly, relearning of the backbone model and deployment of the relearned backbone model to the sensing apparatus can be arranged to be performed when it is determined that an improvement in a configuration of the AI model is required in terms of, for example, accuracy or functionality of inference. Therefore, an improvement in inference performance or accommodation of a required inference function can be achieved.

[0147] Furthermore, in the information processing apparatuses as the embodiments, the predetermined trigger condition is a condition based on an assessment of accuracy of the inference processing performed using the head model. Accordingly, relearning of the backbone model and deployment of the relearned backbone model can be arranged to be performed with a decline in inference accuracy as the trigger condition. Therefore, an improvement in inference performance can be achieved.

[0148] In addition, in the information processing apparatus (information processing apparatus 1E) as one of the embodiments, the feature amount data is associated with environmental estimation basis information that can be used as a basis for estimation with respect to a sensing environment of sensing data used as an extraction source of the feature amount data, and the control unit (control unit F3E) is configured to estimate the sensing environment based on the environmental estimation basis information, select a backbone model in accordance with the estimated sensing environment from among a plurality of candidates of the backbone model, and cause the selected backbone model to be deployed to the sensing apparatus that is a transmission source of the feature amount data. Accordingly, the sensing apparatus is enabled to use the backbone model suitable for the sensing environment. Therefore, an improvement in inference performance can be achieved.

[0149] In addition, an information processing method as an embodiment is an information processing method in which an information processing apparatus: inputs feature amount data from a database storing the feature amount data, the feature amount data having been obtained by a sensing apparatus that includes a sensor and that performs feature amount extraction on sensing data from the sensor using a backbone model being an AI model that performs feature amount extraction on input data; selects, from a plurality of candidates of a head model being an AI model that performs inference processing using feature amount data as input data, the head model based on input information of a user; and performs control using the selected head model so that inference processing using the input feature amount data as input data is performed. Operational advantages similar to those obtained by the information processing apparatuses as the embodiments described above can also be obtained by such an information processing method.

[0150] A program that causes, for example, a CPU, a DSP, and the like or a device including such components to realize functions as the input unit, the selecting unit, the control unit, and the like described earlier with reference to Fig. 8 and the like is conceivable as an embodiment. In other words, a program according to the embodiment is a program readable by a computer apparatus, the program causing the computer apparatus to realize functions of: inputting feature amount data from a database storing the feature amount data, the feature amount data having been obtained by a sensing apparatus that includes a sensor and that performs feature amount extraction on sensing data from the sensor using a backbone model being an AI model that performs feature amount extraction on input data; selecting, from a plurality of candidates of a head model being an AI model that performs inference processing using feature amount data as input data, the head model based on input information of a user; and performing control using the selected head model so that inference processing using the input feature amount data as input data is performed. Such a program enables the functions as the input unit, the selecting unit, the control unit, and the like described earlier to be realized in a device as an information processing apparatus.

[0151] Such a program can be recorded in advance in an HDD (Hard Disc Drive) or an SSD (Solid State Drive) as a recording medium built into a device such as a computer apparatus, a ROM inside a microcomputer including a CPU, and the like. Alternatively, the program can be temporarily or permanently stored (recorded) in a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disc, a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, or a memory card.　Such a removable recording medium can be provided as so-called packaged software. In addition, such a program can be downloaded via a network such as a LAN or the Internet from a download site besides being installed from a removable recording medium to a personal computer or the like.

[0152] In addition, such a program is suitable for making the inference processing method as an embodiment widely available.　Information processing apparatuses in various forms can be caused to function as an apparatus for realizing the inference processing method according to the present disclosure.

[0153] Furthermore, the information processing apparatuses as the embodiments can also be described as including: an input unit configured to input feature amount data from a database storing the feature amount data, the feature amount data having been obtained by a sensing apparatus including a sensor and configured to perform feature amount extraction on sensing data from the sensor using a first AI model configured to perform feature amount extraction on input data; a selecting unit configured to select, from a plurality of candidates of a second AI model configured to perform inference processing using feature amount data as input data, the second AI model based on input information of a user; and a control unit configured to perform control using the second AI model selected by the selecting unit so that inference processing using the feature amount data input by the input unit as input data is performed. In this case, the first AI model can be configured so as to include a third AI model configured to perform feature amount extraction and feature amount integration.　Accordingly, for example, even when the backbone model BB includes the neck model NC described earlier, privacy protection can be achieved while achieving reductions in a storage capacity of the database and an amount of communication data between the sensing apparatus and the database.

[0154] The advantageous effects described in the present specification are merely exemplary and are not restrictive, and other advantageous effects may be produced.

[0155] < 7.　　　Present Technique > The present technique can also be configured as follows. Technique 1.　　　An information processing system comprising: a database configured to store feature data that is generated from sensing data using a backbone model, and processing circuitry configured to: select a head model for inference processing; and cause the selected head model to be used for the inference processing using the feature data that is generated from the sensing data using the backbone model. Technique 2.　　　The information processing system according to technique 1, wherein the processing circuitry is configured to select the head model from among a plurality of different head models. Technique 3.　　　The information processing system according to techniques 1-2, wherein the processing circuitry is configured to select the head model based upon a user input. Technique 4.　　　The information processing system according to techniques 1-3, further comprising a user terminal, wherein the user input is provided by the user terminal. Technique 5.　　　The information processing system according to techniques 1-4, wherein the processing circuitry is configured to select the head model based upon tag information stored in association with each of the plurality of different head models. Technique 6.　　　The information processing system according to techniques 1-5, wherein the processing circuity is configured to select two or more of the plurality of different head models. Technique 7.　　　The information processing system according to techniques 1-6, wherein the processing circuitry is configured to cause each of the two or more of the plurality of different head models to perform respective inference processing simultaneously. Technique 8.　　　The information processing system according to techniques 1-7, wherein the processing circuity is configured to cause the selected head model to be deployed to a sensing apparatus. Technique 9.　　　The information processing system according to techniques 1-8, wherein the processing circuitry is configured to cause the two or more of the plurality of different head models to be deployed to a sensing apparatus. Technique 10.　　　The information processing system according to techniques 1-9, wherein the processing circuitry is configured to issue an instruction to the sensing apparatus to switch between the two or more of the plurality of different head models. Technique 11.　　　The information processing system according to techniques 1-10, wherein the processing circuitry is configured to cause the backbone model to be deployed to a sensing apparatus, and the sensing apparatus generates the sensing data. Technique 12.　　　The information processing system according to techniques 1-11, wherein the processing circuitry is configured to select the backbone model from among a plurality of different backbone models according to ancillary information obtained by the sensing apparatus. Technique 13.　　　The information processing system according to techniques 1-12, wherein the processing circuitry is configured to cause a plurality of different backbone models to be deployed to the sensing apparatus. Technique 14.　　　The information processing system according to techniques 1-13, wherein the processing circuitry is configured to issue an instruction to the sensing apparatus to switch between the plurality of different backbone models. Technique 15.　　　The information processing system according to techniques 1-14, wherein the processing circuitry is configured to cause the selected head model to be transferred from the sensing apparatus to another sensing apparatus. Technique 16.　　　The information processing system according to techniques 1-15, wherein the processing circuitry is configured to cause a sensing apparatus to transfer the feature data to another sensing apparatus, and the sensing apparatus generates the sensing data and accommodates the backbone model. Technique 17.　　　The information processing system according to techniques 1-16, wherein the processing circuitry is configured to execute relearning processing of the backbone model based upon a relearning condition. Technique 18.　　　The information processing system according to techniques 1-17, wherein the relearning condition includes an error in the inference processing using the head model. Technique 19.　　　A method comprising storing, in a database, feature data that is generated from sensing data using a backbone model; selecting a head model for inference processing; and causing the selected head model to be used for the inference processing using the feature data that is generated from the sensing data using the backbone model. Technique 20　　　A non-transitory computer readable medium storing instructions which when execute by a computer cause the computer to perform a method, the method comprising storing, in a database, feature data that is generated from sensing data using a backbone model; selecting a head model for inference processing; and causing the selected head model to be used for the inference processing using the feature data that is generated from the sensing data using the backbone model.

[0156] 1, 1A, 1B, 1C, 1D, 1E Information processing apparatus 2 Network 3 Sensing apparatus 4 User terminal 5 Database 5a Accumulated information 11, 11A, 11B, 11C CPU 12 ROM 13 RAM 14 AI processing unit 30 Image sensor 31 Imaging optical system 32 Optical system drive unit 33 Camera control unit 34 Memory unit 35 Communicating unit 36 Sensor unit 37 Bus 41 Imaging unit 42 Image signal processing unit 43 In-sensor control unit 44 AI processing unit 45 Memory unit BB, BB1, BB2, BB’, BBx Backbone model HD, HDh, HDc, HD1, HD2 Head model F1 Input unit F2 Selecting unit F3, F3A, F3B, F3C, F3D, F3E Control unit F4 Model storage unit F5 Analyzing unit F6 Presentation processing unit F7 Relearning processing unit F31 Environment estimating unit F32 Model selecting unit F33 Deployment processing unit

Claims

1. An information processing system comprising: a database configured to store feature data that is generated from sensing data using a backbone model, and processing circuitry configured to: select a head model for inference processing; and cause the selected head model to be used for the inference processing using the feature data that is generated from the sensing data using the backbone model.

2. The information processing system according to claim 1, wherein the processing circuitry is configured to select the head model from among a plurality of different head models.

3. The information processing system according to claim 2, wherein the processing circuitry is configured to select the head model based upon a user input.

4. The information processing system according to claim 3, further comprising a user terminal, wherein the user input is provided by the user terminal.

5. The information processing system according to claim 2, wherein the processing circuitry is configured to select the head model based upon tag information stored in association with each of the plurality of different head models.

6. The information processing system according to claim 2, wherein the processing circuity is configured to select two or more of the plurality of different head models.

7. The information processing system according to claim 6, wherein the processing circuitry is configured to cause each of the two or more of the plurality of different head models to perform respective inference processing simultaneously.

8. The information processing system according to claim 1, wherein the processing circuity is configured to cause the selected head model to be deployed to a sensing apparatus.

9. The information processing system according to claim 6, wherein the processing circuitry is configured to cause the two or more of the plurality of different head models to be deployed to a sensing apparatus.

10. The information processing system according to claim 9, wherein the processing circuitry is configured to issue an instruction to the sensing apparatus to switch between the two or more of the plurality of different head models.

11. The information processing system according to claim 1, wherein the processing circuitry is configured to cause the backbone model to be deployed to a sensing apparatus, and the sensing apparatus generates the sensing data.

12. The information processing system according to claim 11, wherein the processing circuitry is configured to select the backbone model from among a plurality of different backbone models according to ancillary information obtained by the sensing apparatus.

13. The information processing system according to claim 11, wherein the processing circuitry is configured to cause a plurality of different backbone models to be deployed to the sensing apparatus.

14. The information processing system according to claim 13, wherein the processing circuitry is configured to issue an instruction to the sensing apparatus to switch between the plurality of different backbone models.

15. The information processing system according to claim 8, wherein the processing circuitry is configured to cause the selected head model to be transferred from the sensing apparatus to another sensing apparatus.

16. The information processing system according to claim 1, wherein the processing circuitry is configured to cause a sensing apparatus to transfer the feature data to another sensing apparatus, and the sensing apparatus generates the sensing data and accommodates the backbone model.

17. The information processing system according to claim 1, wherein the processing circuitry is configured to execute relearning processing of the backbone model based upon a relearning condition.

18. The information processing system according to claim 17, wherein the relearning condition includes an error in the inference processing using the head model.

19. A method comprising: storing, in a database, feature data that is generated from sensing data using a backbone model; selecting a head model for inference processing; and causing the selected head model to be used for the inference processing using the feature data that is generated from the sensing data using the backbone model.

20. A non-transitory computer readable medium storing instructions which when execute by a computer cause the computer to perform a method, the method comprising: storing, in a database, feature data that is generated from sensing data using a backbone model; selecting a head model for inference processing; and causing the selected head model to be used for the inference processing using the feature data that is generated from the sensing data using the backbone model.

Images