Information processing device, information processing method, and program

Abstract

Provided are an information processing device and the like which can estimate the number of occupants in a vehicle even if it is difficult to estimate the number of occupants from a captured image. An information processing device according to an embodiment of the present disclosure is provided with: a determination means which determines whether or not the number of occupants in a vehicle can be estimated from a captured image of the vehicle; and a first estimation means which, if it is determined that the number of occupants cannot be estimated, estimates the number of occupants on the basis of information relating to a signal detected from a region containing the vehicle.

Description

[Technical field]

[0001] The present disclosure relates to a technique for utilizing signals output from a communication device. [Background technology]

[0002] Technology exists to estimate the number of occupants in vehicles traveling on a road.

[0003] For example, Patent Document 1 discloses a technique for estimating the number of passengers in a vehicle by performing processing for detecting people on an image captured by a camera and integrating the person detection results. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2014 / 064898 Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology disclosed in Patent Document 1, the number of passengers in a vehicle is estimated based only on information obtained from a captured image. Therefore, for example, in cases where the vehicle is not clearly visible in the captured image due to bad weather such as fog or snowfall, or when the captured image cannot be reliably acquired due to some abnormality occurring in the camera, it may be difficult to estimate the number of passengers.

[0006] The present disclosure has been made in consideration of the above-mentioned problems, and one of its objectives is to provide an information processing device, etc. that can estimate the number of passengers in a vehicle even when it is difficult to estimate the number of passengers from a captured image. [Means for solving the problem]

[0007] An information processing device of one embodiment of the present disclosure includes a determination means for determining whether or not the number of occupants in a vehicle can be estimated from an image captured of the vehicle, and a first estimation means for estimating the number of occupants based on information relating to a signal detected from an area including the vehicle if it is determined that the number of occupants cannot be estimated.

[0008] An information processing method of one embodiment of the present disclosure determines whether the number of occupants in a vehicle can be estimated from an image of the vehicle, and if it is determined that the number of occupants cannot be estimated, estimates the number of occupants based on information related to a signal detected from an area including the vehicle.

[0009] A computer-readable storage medium of one embodiment of the present disclosure stores a program that causes a computer to execute the following processes: determining whether or not the number of occupants in a vehicle can be estimated from an image of the vehicle; and, if it is determined that the number of occupants cannot be estimated, estimating the number of occupants based on information regarding signals detected from an area including the vehicle. Effect of the Invention

[0010] According to the present disclosure, even when it is difficult to estimate the number of vehicle occupants from a captured image, the number of occupants can be estimated. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating an example of a configuration of an information processing system according to a first embodiment of the present disclosure. [Diagram 2] 1 is a block diagram including an example of a functional configuration of an information processing device according to a first embodiment of the present disclosure. [Diagram 3] FIG. 2 is a diagram illustrating an example of a database according to the first embodiment of the present disclosure. [Figure 4] 5 is a flowchart illustrating an example of an operation of the information processing device according to the first embodiment of the present disclosure. [Diagram 5] FIG. 11 is a block diagram including an example of a functional configuration of an information processing device according to a second embodiment of the present disclosure. [Figure 6] 10 is a flowchart illustrating an example of an operation of the information processing device according to the second embodiment of the present disclosure. [Figure 7] FIG. 13 is a block diagram including an example of a functional configuration of an information processing device according to a third embodiment of the present disclosure. [Figure 8] 13 is a flowchart showing an example of an operation of the information processing device according to the third embodiment of the present disclosure. [Figure 9] FIG. 13 is a block diagram including an example of a functional configuration of an information processing device according to a fourth embodiment of the present disclosure. [Figure 10] FIG. 13 is a block diagram showing an example of a functional configuration of an information processing device according to a fifth embodiment of the present disclosure. [Figure 11] 13 is a flowchart showing an example of an operation of an information processing device according to the fifth embodiment of the present disclosure. [Figure 12] FIG. 2 is a block diagram showing an example of a hardware configuration of a computer device that realizes an information processing device according to the first, second, third, fourth, and fifth embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0013] <First embodiment> An information processing system including an information processing device according to a first embodiment will be described.

[0014] FIG. 1 is a diagram showing a schematic example of a configuration of an information processing system 1000 according to a first embodiment. As shown in FIG. 1, the information processing system 1000 includes an information processing device 100, an image capturing device 200, and a sensor 300. The information processing device 100 is connected to the image capturing device 200 and the sensor 300 in a wired or wireless manner so as to be able to communicate with each other. The image capturing device 200 and the sensor 300 may be an integrated device. The information processing device 100 may be an integrated device with either or both of the image capturing device 200 and the sensor 300. As shown in FIG. 1, the information processing device 100 may be connected to a storage device 400 and a terminal 500 in a wired or wireless manner so as to be able to communicate with each other. The storage device 400 may be mounted on the information processing device 100. The terminal 500 is a device equipped with an input / output means for communicating with the information processing device 100 via a network. The terminal 500 may be a personal computer or a portable terminal such as a smartphone or a tablet terminal.

[0015] In the situation shown in FIG. 1, the image capturing device 200 captures an image of a vehicle. The sensor 300 detects a predetermined signal in a range including the range captured by the image capturing device 200. The image capturing device 200 and the sensor 300 may be installed in multiple locations. The information processing device 100 estimates the number of passengers in the vehicle based on information related to the detected signal. Here, one or more people are in the vehicle. The information processing device 100 can calculate the number of communication devices by acquiring information related to signals output from communication devices such as smartphones carried by the people. Then, the information processing device 100 estimates the number of passengers in the vehicle based on the calculated number of communication devices.

[0016] Next, the configuration of the information processing system 1000 will be described in detail.

[0017] [Details of the imaging device 200] The photographing device 200 photographs a predetermined range and generates photographing data. In this specification, the range in real space photographed by the photographing device 200 is also referred to as the photographing range, and the image photographed by the photographing device 200 is also referred to as the photographed image. The photographed image may be a still image or any of a plurality of frames constituting a moving image. If a vehicle exists in the photographing range, the vehicle is reflected in the photographed image. The photographing data indicates information constituting the photographed image. The photographing data may also be data in which the photographed image is encoded. Then, the photographing device 200 transmits the photographing data to the information processing device 100. The information processing device 100 obtains the photographed image by, for example, decoding the photographing data received from the photographing device 200. The photographing data may include information identifying the photographing device 200. Furthermore, the photographing data may include information on the time of photographing, information indicating the photographing location, and information identifying the photographing device 200. In this case, the photographing device 200 has a function of calculating time and a storage unit (not shown) that stores information indicating the photographing location.

[0018] The photographing device 200 may continuously photograph the photographing range and sequentially transmit the photographed data to the information processing device 100. Also, a moving object sensor (not shown) communicably connected to the photographing device 200 may be installed around the photographing range. Then, when the moving object sensor detects a moving object, that is, when it detects the passage of a vehicle, the photographing device 200 may perform photographing.

[0019] [Sensor 300 Details] The sensor 300 detects a signal output from a communication device present within a predetermined range, and generates detection data. In this specification, the range in real space in which the sensor 300 detects a signal is also referred to as a detection range. The communication device that outputs the signal is, for example, a mobile phone, a smartphone, a tablet terminal, or the like, but is not limited to this example. The sensor 300 can detect a signal, for example, in a range that includes the shooting range. In addition, the sensor 300 is, for example, a single sensor that can detect a signal in a predetermined frequency band, but the example of the sensor 300 is not limited to this example. For example, the sensor 300 may be composed of multiple sensors. In this case, the multiple sensors may each be a sensor that can detect a signal in a different frequency band. The detection data is information indicating a signal detected by the sensor 300. The sensor 300 transmits the detection data to the information processing device 100.

[0020] The detected signal may be radio waves output from a communication device. That is, the sensor 300 can detect radio waves output from a communication device present in the detection range, for example, and generate detection data. The communication device may be a device that outputs radio waves of any communication method. In this case, the detection data includes information indicating the signal waveform of the detected radio waves. The information indicating the signal waveform of the radio waves is, for example, information indicating the amplitude value of the radio wave signal in time series. The radio waves detected by the sensor 300 may be, for example, radio waves of LTE (Long Term Evolution), wireless LAN (Local Area Network), short-distance wireless, etc. In addition, when the communication device is a smartphone or the like, the timing of outputting radio waves is, for example, during a voice call, during communication due to the operation of an application, or during polling communication between the communication device and a base station.

[0021] The sensor 300 transmits the detection data to the information processing device 100. At this time, the detection data may include identification information of the sensor 300. Furthermore, the detection data may include information indicating the time when the radio wave was detected, information indicating the location where the sensor 300 is installed, and information for identifying the sensor 300. In this case, the sensor 300 has a function of calculating the time and a storage unit (not shown) that stores information indicating the location.

[0022] Furthermore, the sensor 300 may be controlled by the information processing device 100. For example, the sensor 300 may start and stop detecting a signal in response to an instruction from the information processing device 100. Furthermore, the sensor 300 may change its detection range in response to an instruction from the information processing device 100.

[0023] [Details of the information processing device 100] Next, details of the information processing device 100 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of a functional configuration of the information processing device 100. As shown in Fig. 2, the information processing device 100 includes a determination unit 110, a first estimation unit 120, an extraction unit 130, a second estimation unit 140, and a registration unit 150.

[0024] The determination unit 110 determines whether or not the number of passengers can be estimated from the captured image. For example, the determination unit 110 acquires image capture environment data, which is data that may affect image capture performed by the image capture device 200. Then, the determination unit 110 determines whether or not the number of passengers can be estimated based on the image capture environment data. For example, the determination unit 110 acquires weather data of the location where the image capture device 200 is installed as the image capture environment data via the Internet. Then, the determination unit 110 may determine that the number of passengers cannot be estimated from the captured image when it is raining, snowing, or foggy at the location where the image capture device 200 is installed. At this time, the determination unit 110 may determine that the number of passengers cannot be estimated from the captured image when the weather data includes information indicating a warning or alert such as heavy rain, heavy snow, and dense fog.

[0025] Alternatively, the determination unit 110 may determine that the number of passengers cannot be estimated from the captured image by acquiring information about an abnormality in the image capturing device 200 as image capturing environment data. For example, assume that a lens failure, a power supply failure, or the like occurs in the image capturing device 200. At this time, the image capturing device 200 notifies the information processing device 100 that a failure has occurred. The determination unit 110 may determine that the number of passengers cannot be estimated from the captured image by regarding the notification as information about the abnormality. Note that the determination unit 110 may acquire information about the abnormality in the image capturing device 200 by a user inputting the information through the terminal 500.

[0026] The captured image may have whiteout and blackout, or may be blurred due to vibration or adhesions on the lens. In this way, if the quality of the captured image is poor, there is a risk that the number of passengers cannot be accurately estimated from the captured image. Therefore, the determination unit 110 may obtain information on the quality of the captured image as the shooting environment data, and determine whether the number of passengers can be estimated from the captured image based on the information on the quality. The information on the quality is information that can be obtained from the captured image. For example, the information on the quality is an index on the histogram of pixel values ​​of the captured image or the amount of edges detected from the captured image, but is not limited to this example. The information on the quality may be the result of performing various image processing on the captured image. For example, the determination unit 110 obtains a histogram of pixel values ​​from the captured image, and determines that whiteout or blackout has occurred in the captured image based on the bias of the histogram. Then, the determination unit 110 may determine that the number of passengers cannot be estimated from the captured image when whiteout or blackout has occurred in the captured image. Also, for example, the determination unit 110 acquires an index relating to the amount of edges detected from the captured image, and when the index is less than a predetermined threshold, determines that the amount of edges detected from the captured image is small, that is, the captured image is blurred. Then, when the captured image is blurred, the determination unit 110 may determine that the number of passengers cannot be estimated from the captured image.

[0027] In this manner, the determination unit 110 determines whether or not the number of passengers in the vehicle can be estimated from the captured image of the vehicle. The determination unit 110 is an example of a determination means.

[0028] When it is determined that the number of passengers cannot be estimated from the captured image, the first estimation unit 120 estimates the number of passengers in the vehicle based on information about signals detected from a range including the vehicle. The information about the signal is, for example, information for identifying the communication device that is included in the signal output from the communication device. The first estimation unit 120 acquires information for identifying the communication device that is extracted from information indicating the signal detected by the sensor 300 (i.e., detection data). The first estimation unit 120 then calculates the number of communication devices that output signals using the information for identifying the communication devices, and estimates the number of passengers in the vehicle based on the calculated number of communication devices.

[0029] Here, if the detection data includes information indicating the signal waveform of the radio wave detected by the sensor 300, the first estimation unit 120 may use the radio wave characteristic information as information for identifying the communication device. The radio wave characteristic information is information extracted from the information indicating the signal waveform of the radio wave, and is information indicating the characteristics of the signal waveform of the radio wave that differ for each communication device that outputs radio waves. The first estimation unit 120 may calculate the number of communication devices based on the radio wave characteristic information.

[0030] Although communication devices that output radio waves are manufactured to meet a predetermined communication standard, minute individual differences exist due to variations in analog circuits, etc. Therefore, radio waves have features that can be identified based on the individual differences of the devices that are the source of the radio waves. The radio wave characteristic information may be, for example, a feature based on a modulation error obtained by modulation analysis, or a feature based on the time waveform of the radio waves. In addition, for example, the radio wave characteristic information may be a feature indicating one or more of the transient (rising and falling) of the received signal, the power spectrum density of the preamble portion, the error vector magnitude (EVM), the IQ (In-phase Quadrature-phase) phase error, the IQ imbalance amount, the frequency offset, and the symbol clock error, but is not limited to these.

[0031] Since radio wave characteristic information depends on the hardware of the communication device, it is difficult to rewrite it manually. Therefore, radio wave characteristic information is unlikely to be intentionally spoofed. In addition, radio wave characteristic information is information that can identify individual communication devices regardless of the type of information transmitted by radio waves, such as the mobile phone carrier.

[0032] The process of extracting the radio wave characteristic information from the information indicating the signal waveform of the radio wave is performed by the extraction unit 130 according to a predetermined rule, for example. Here, the extraction unit 130 can use machine learning when extracting the radio wave characteristic information from the information indicating the signal waveform of the radio wave. Specifically, the extraction unit 130 can extract the radio wave characteristic information from the information indicating the signal waveform of the radio wave by using an extraction model generated based on the learning data. For example, the learning data includes information indicating the signal waveform of the radio wave output from various communication devices and radio wave characteristic information corresponding to each of the information indicating the signal waveform of the radio wave. By using such learning data, an extraction model can be generated in which the information indicating the signal waveform of the radio wave is input and the radio wave characteristic information is output. The learning data and the extraction model may be stored in the storage device 400 in advance. The generation of the learning model may be performed by the extraction unit 130. In this way, the extraction unit 130 extracts information identifying the communication device from the signal detected from the range including the vehicle. The extraction unit 130 is an example of an extraction means.

[0033] The first estimation unit 120 calculates the number of communication devices present within the detection range based on, for example, the type or number of radio wave characteristic information. The first estimation unit 120 can estimate, for example, the calculated number of communication devices as the number of passengers in the vehicle. The calculated number of communication devices does not have to be the number of passengers as it is. For example, when the calculated number of communication devices is three, the first estimation unit 120 may estimate that the number of passengers is two to four. That is, the first estimation unit 120 may output a value within a predetermined range as the estimated number of passengers. The first estimation unit 120 may store information indicating the estimation result in the storage device 400 or output it to the terminal 500.

[0034] In this way, when it is determined that the number of passengers cannot be estimated, the first estimation unit 120 estimates the number of passengers based on information about signals detected from a range including the vehicle. The first estimation unit 120 is an example of a first estimation means.

[0035] The second estimation unit 140 estimates the number of passengers in the vehicle based on the captured image. For example, the second estimation unit 140 estimates the number of passengers by detecting people from the captured image captured by the image capture device 200 and calculating the number of detected people. A method for detecting people from the captured image may be a method of detecting the shape of the person's face or body, or a method of detecting people from the captured image by performing pattern matching using an image of a person. When performing pattern matching, the image of the person may be stored in advance in, for example, the storage device 400. In addition, a method for estimating the number of passengers from a captured image may be, for example, a method described in Patent Document 1. The second estimation unit is an example of a second estimation means.

[0036] The registration unit 150 registers data in which information about the traffic light, the estimated number of passengers, and the captured image are associated with each other in a database. The database is stored in the storage device 400, for example. When there is a plurality of pieces of information about the traffic light, the registration unit 150 associates the information about the plurality of traffic lights, the estimated number of passengers, and the captured image. The image associated with the information about the traffic light, etc. may be an image obtained by processing an image captured by the image capture device 200. For example, the registration unit 150 may register an image in which an area showing a vehicle is cut out from the image captured by the image capture device 200 in association with the information about the traffic light, etc. in the database. The registration unit 150 may further associate information about the vehicle extracted from the captured image with each other and register the information in the database. For example, the registration unit 150 may read the license plate of the vehicle from the captured image, and register the read license plate information in association with the information about the traffic light, etc. in the database. The process of reading the license plate may be performed by, for example, the extraction unit 130. The process of reading the license plate may be performed by an external device (not shown) connected to the information processing device 100. The registration unit 150 may further associate the number of passengers of the vehicle shown in the captured image. For example, the extraction unit 130 may estimate the number of passengers of the vehicle from the size and shape of the vehicle shown in the captured image.

[0037] FIG. 3 is a diagram showing an example of a database. In this example, radio wave characteristic information is set as information about a signal. In the record in the first row of FIG. 6, "data A" and "data B" which are radio wave characteristic information, data 1 which is a photographed image, and "2" which indicates the number of passengers are associated with each other. Also, as shown in FIG. 6, the registration unit 150 may further associate time and location information with the data in which the information about the signal, the photographed image, and the number of passengers are associated with each other. The time information may be, for example, either the time when the photographing device 200 photographed the image or the time when the sensor 300 detected the signal, or the time when the information processing device 100 acquired the photographed data or the detected data. The location information may be, for example, information on the installation location of the photographing device 200 included in the photographed data, or information on the installation location of the sensor 300 included in the detected data. Note that, when the photographed data and the detected data do not include location information, the registration unit 150 may acquire location information using information for identifying the photographing device 200 included in the photographed data, or information for identifying the sensor 300 included in the detected data. In this case, a database in which information for identifying the image capturing device 200 is associated with information on the installation location of the image capturing device 200, or a database in which information for identifying the sensor 300 is associated with information on the installation location of the sensor 300, is stored in advance in the storage device 400. For example, the registration unit 150 identifies, from the database, information on the location associated with information for identifying the sensor 300, which is included in the detection data.

[0038] In this way, the registration unit 150 associates the information about the signal, the estimated number of passengers, and the captured image with each other and registers them in the database. The registration unit 150 is an example of a registration means.

[0039] The registration unit 150 may register data associating any two of the information on the traffic signal, the number of passengers, and the captured image. For example, the registration unit 150 may register data associating the information on the traffic signal with the captured image, or may register data associating the information on the traffic signal with the number of passengers.

[0040] [Operation of information processing device 100] Next, the operation of the information processing device 100 will be described with reference to Fig. 4. In this operation example, it is assumed that the information processing device 100 acquires a captured image and detection data. At this time, it is assumed that the detection data includes information indicating the signal waveform of the radio wave detected by the sensor 300. Also, in this operation example, it is assumed that the information processing device 100 acquires weather data as the image capturing environment data. Note that in this specification, each step of the flowchart is expressed using a number assigned to each step, such as "S101".

[0041] 4 is a flowchart for explaining an example of the operation of the information processing device 100. First, the determination unit 110 determines whether or not the number of passengers can be estimated from the captured image based on the image capture environment data (S101). If it is determined that the number of passengers cannot be estimated from the captured image ("No" in S102), the extraction unit 130 extracts radio wave characteristic information from information indicating the signal waveform of radio waves included in the detection data (S103). Then, the first estimation unit 120 estimates the number of passengers based on the extracted radio wave characteristic information (S104). The registration unit 150 associates the radio wave characteristic information, the captured image, and the estimated number of passengers and registers them in a database (S105).

[0042] If the result of the process in S102 is "Yes," that is, if it is determined that the number of passengers can be estimated from the captured image, the second estimation unit 140 estimates the number of passengers based on the captured image (S106), and then performs the process in S105.

[0043] In addition, when the information processing device 100 acquires information on an abnormality of the photographing device 200 as the photographing environment data, there is a possibility that a photographed image cannot be obtained. In this case, in the process of S105, the registration unit 150 may register the radio wave characteristic information and the number of passengers in the database in association with each other.

[0044] In this way, the information processing device 100 of the first embodiment determines whether or not the number of passengers in the vehicle can be estimated from a captured image of the vehicle, and if it is determined that the number of passengers cannot be estimated, estimates the number of passengers based on information about signals detected from a range including the vehicle. This allows the information processing device 100 to estimate the number of passengers in the vehicle without using a captured image of the vehicle. That is, the information processing device 100 of the first embodiment has the effect of being able to estimate the number of passengers even when it is difficult to estimate the number of passengers in the vehicle from a captured image.

[0045] The information processing system 1000 in the present disclosure can be used in combination with, for example, a speed measurement device installed on a road. The speed measurement device is a device that measures the speed of a vehicle and determines whether the measured speed exceeds a predetermined threshold. For example, the information processing system 1000 can register in a database a captured image of a vehicle whose speed is determined to exceed a threshold and information about a traffic light in association with each other. This allows the information processing system 1000 to present information for identifying a person who was in a vehicle traveling at a speed exceeding the threshold.

[0046] The information processing system 1000 in the present disclosure can also be used to monitor road usage. For example, there are lanes on which only vehicles with a specified number of passengers or more can travel. Lanes on which only vehicles with a specified number of passengers or more can travel are also called HOV (High Occupancy Vehicles) lanes, carpool lanes, or ride-sharing lanes. The information processing system 1000 can estimate the number of passengers in a vehicle, and therefore can present data for monitoring whether such lanes are being used correctly.

[0047] [Variation 1] In the above embodiment, an example has been described in which the information processing device 100 estimates the number of passengers by using radio wave characteristic information as information related to a signal. However, the information related to a signal is not limited to this example.

[0048] For example, a communication device such as a smartphone or a tablet terminal periodically transmits a signal to check whether or not communication with a surrounding access point is possible. The signal transmitted at this time is also called a probe request signal. The sensor 300 may detect the probe request signal and generate detection data including information indicating the probe request signal.

[0049] In this case, when the extraction unit 130 receives the detection data, it extracts the source MAC (Media Access Control) address included in the probe request signal. The first estimation unit 120 uses the extracted MAC addresses as identifiers of the communication devices, and calculates the number of communication devices present within the detection range based on the number of the identifiers. The first estimation unit 120 then estimates the number of passengers based on the calculated number of communication devices.

[0050] [Variation 2] In the above embodiment, an example has been described in which the extraction unit 130 of the information processing device 100 extracts information identifying a communication device from information indicating a signal detected by the sensor 300. The process of extracting the information identifying the communication device may be performed by the sensor 300.

[0051] It is assumed that the information for identifying the communication device is radio wave characteristic information. In this case, when the sensor 300 detects radio waves within the detection range, it extracts the radio wave characteristic information from the signal waveform of the detected radio waves. The sensor 300 transmits detection data including the extracted radio wave characteristic information to the information processing device 100. In this case, the extraction model may be stored in advance in a storage device (not shown) included in the sensor 300.

[0052] When the information for identifying the communication device is a MAC address, the sensor 300 extracts the MAC address from the detected probe request signal, for example, and transmits detection data including information indicating the extracted MAC address to the information processing device 100.

[0053] The first estimation unit 120 of the information processing device 100 estimates the number of passengers in the vehicle based on the information for identifying the communication device included in the detection data. In this modified example, the information processing device 100 does not need to perform the process of S103 in the flowchart of FIG.

[0054] [Variation 3] The information processing device 100 may acquire the number of passengers estimated by the second estimation unit as the shooting environment data. That is, the shooting environment data may be the number of passengers estimated based on the captured image. At this time, the determination unit 110 may determine that the number of passengers cannot be estimated from the captured image when the number of passengers is a predetermined value. For example, when an unnatural number of passengers is estimated as the number of passengers in one vehicle, such as the number of passengers being estimated to be 0 or 100, the determination unit 110 may determine that the number of passengers cannot be estimated from the captured image. Note that the criterion for the unnatural number of passengers is not limited to this example, and any numerical value may be set.

[0055] 4, the second estimation unit 140 performs processing of S106 before processing of S101, that is, processing of estimating the number of passengers based on the captured image. If the determination unit 110 determines that the number of passengers cannot be estimated from the captured image ("No" in S102), the information processing device 100 proceeds to processing of S103. If the determination unit 110 determines that the number of passengers can be estimated from the captured image ("Yes" in S102), the information processing device 100 proceeds to processing of S105.

[0056] <Second embodiment> Next, an information processing system including an information processing device according to a second embodiment will be described. In the second embodiment, an example will be described in which a sensor detects a signal when it is determined that the number of passengers cannot be estimated from a captured image.

[0057] Fig. 5 is a block diagram showing an example of the configuration of an information processing system 1001 according to the second embodiment. As shown in Fig. 5, the information processing system 1001 includes an information processing device 101 instead of the information processing device 100 in the first embodiment, and is otherwise similar to the information processing system 1000 described in the first embodiment. That is, the information processing system 1001 includes the information processing device 101, an imaging device 200, and a sensor 300. Note that a description of the configuration and operation of the information processing system 1001 shown in Fig. 5 that overlap with the description of the first embodiment will be omitted.

[0058] [Details of information processing device 101] As shown in FIG. 5, the information processing device 101 includes a determination unit 110, a first estimation unit 120, an extraction unit 130, a second estimation unit 140, a registration unit 150, and a control unit 160.

[0059] The control unit 160 controls the sensor 300. For example, when the determination unit 110 determines that the number of passengers cannot be estimated from the captured image, the control unit 160 instructs the sensor 300 to start detecting signals. Upon receiving the instruction, the sensor 300 detects signals within a detection range. The control unit 160 may determine the detection range of the sensor 300.

[0060] In this way, when it is determined that the number of passengers cannot be estimated, the control unit 160 controls the sensor to detect the signal from a range including the vehicle. The control unit 160 is an example of a control means.

[0061] Note that, when the determination unit 110 determines that the number of passengers can be estimated from the captured image, the control unit 160 may instruct the sensor 300 to stop detecting the signal. In this case, upon receiving the instruction, the sensor 300 stops detecting the signal.

[0062] [Operation of information processing device 101] Next, the operation of the information processing device 101 will be described with reference to Fig. 6. Fig. 6 is a flowchart illustrating an example of the operation of the information processing device 101. In this operation example, it is assumed that the sensor 300 detects a signal waveform of a radio wave.

[0063] The processes of S201 and S202 are the same as those of S101 and S102 in Fig. 4. When it is determined that the number of passengers cannot be estimated from the captured image ("No" in S202), the control unit 160 causes the sensor 300 to start detecting radio waves (S203). When the extraction unit 130 acquires detection data from the sensor 300 (S204), it extracts radio wave characteristic information from information indicating the signal waveform of radio waves included in the detection data (S205). Then, the first estimation unit 120 estimates the number of passengers based on the extracted radio wave characteristic information (S206). The registration unit 150 associates the radio wave characteristic information, the captured image, and the estimated number of passengers and registers them in a database (S207).

[0064] If the result of the process in S202 is "Yes," that is, if it is determined that the number of passengers can be estimated from the captured image, the second estimation unit 140 estimates the number of passengers based on the captured image (S208). Then, the registration unit 150 associates the captured image with the estimated number of passengers and registers them in the database (S209).

[0065] In this way, when it is determined that the number of passengers cannot be estimated, the information processing device 101 of the second embodiment controls the sensor to detect signals from a range including the vehicle. This allows the information processing device 101 to operate the sensor 300 only when necessary, for example. Therefore, the information processing device 101 can reduce the power consumption of the system.

[0066] <Third embodiment> Next, an information processing system including an information processing device according to a third embodiment will be described. In the third embodiment, an example will be described in which information related to a signal registered in a database is compared with information related to a signal detected by a sensor.

[0067] Fig. 7 is a block diagram showing an example of the configuration of an information processing system 1002 according to the third embodiment. As shown in Fig. 7, the information processing system 1002 includes an information processing device 102 instead of the information processing device 100 in the first embodiment, and is otherwise similar to the information processing system 1000 described in the first embodiment. That is, the information processing system 1002 includes the information processing device 102, an imaging device 200, and a sensor 300. Note that a description of the configuration and operation of the information processing system 1002 shown in Fig. 7 that overlap with the description of the first and second embodiments will be omitted.

[0068] [Details of the information processing device 102] As shown in FIG. 7, the information processing device 102 includes a determination unit 110, a first estimation unit 120, an extraction unit 130, a second estimation unit 140, a registration unit 150, a control unit 160, a matching unit 170, and an output unit 180.

[0069] The matching unit 170 matches information about the signal detected by the sensor 300 with information about the signal registered in the database. For example, assume that radio wave characteristic information is registered in the database as information about the signal. In this case, the extraction unit 130 extracts the radio wave characteristic information from the detection data. Then, the matching unit 170 matches the extracted radio wave characteristic information with the radio wave characteristic information registered in the database.

[0070] Here, the matching unit 170 can use machine learning when matching the information related to the signal. Specifically, the matching unit 170 uses a matching model generated based on learning data to determine whether the matching is a match. The matching model is, for example, a model for matching the extracted information related to the signal with the registered information related to the signal and determining whether they match. By inputting the extracted information related to the signal to the matching model, the similarity between the extracted information related to the signal and the registered information related to the signal is calculated. If the similarity is equal to or greater than a predetermined value, the matching unit 170 determines that the matching is a match.

[0071] In this way, the collation unit 170 collates the information on the signal detected from the range including the vehicle with the information on the signal registered in the database. The collation unit 170 is an example of a collation means.

[0072] The output unit 180 outputs the result of the matching by the matching unit 170. For example, when the matching by the matching unit 170 is a match, the output unit 180 transmits information indicating that the matching is a match to the terminal 500, and outputs information indicating that the matching is a match at the terminal 500. At this time, the output unit 180 may output at least one of the time when the signal was detected, the location where the signal was detected, the captured image of the location where the signal was detected, and the estimated number of passengers to the terminal 500. Furthermore, when the matching is a match, the output unit 180 may output information of a record including information on the signal used in the matching to the terminal 500. At this time, the output unit 180 may output at least one of the information included in the record. For example, when the record of the database shown in FIG. 3 is specified, the output unit 180 may output at least one of the information of the radio wave characteristic information, the captured image, the number of passengers, the time, and the location. Furthermore, when the matching by the matching unit 170 is not a match, the output unit 180 may output information indicating that the matching is not a match to the terminal 500. The terminal 500 displays the information received from the output unit 180 on, for example, a display that the terminal 500 has.

[0073] [Operation of information processing device 102] Next, the operation of the information processing device 102 will be described with reference to Fig. 8. Fig. 8 is a flowchart for explaining an example of the operation of the information processing device 102. Note that in this operation example, it is assumed that at least one record is registered in the database by the operation shown in Fig. 4. Also, it is assumed that the sensor 300 detects radio waves and generates detection data including information indicating the signal waveform of the detected radio waves.

[0074] First, the extraction unit 130 extracts radio wave characteristic information from information indicating the signal waveform of the radio wave (S301). The comparison unit 170 compares the extracted radio wave characteristic information with radio wave characteristic information registered in the database (S302). If the comparison matches ("Yes" in S303), the output unit 180 outputs the comparison result to the terminal 500 (S304). For example, the output unit 180 outputs the location and time at which the radio wave, including the radio wave characteristic information, was detected, and an image captured at the location at which the radio wave was detected. If the comparison does not match ("No" in S203), the flow ends.

[0075] In this way, the information processing device 102 of the third embodiment compares information about signals detected in a range including the vehicle with information about signals registered in the database. This allows the information processing device 102 to detect a specific person and a vehicle in which the specific person is riding, using the information about signals.

[0076] <Fourth embodiment> Next, a description will be given of an information processing system including an information processing device according to a fourth embodiment. In the fourth embodiment, an example in which an alarm is generated according to an estimated number of passengers will be described.

[0077] Fig. 9 is a block diagram showing an example of the configuration of an information processing system 1003 according to the fourth embodiment. As shown in Fig. 7, the information processing system 1003 includes an information processing device 103 instead of the information processing device 100 in the first embodiment, and is otherwise similar to the information processing system 1000 described in the first embodiment. That is, the information processing system 1003 includes the information processing device 103, an imaging device 200, and a sensor 300. Note that a description of the configuration and operation of the information processing system 1002 shown in Fig. 9 that overlap with the description of the first, second, and third embodiments will be omitted.

[0078] [Details of the information processing device 103] As shown in FIG. 9, the information processing device 103 includes a determination unit 110, a first estimation unit 120, an extraction unit 130, a second estimation unit 140, a registration unit 150, a control unit 160, a comparison unit 170, an output unit 180, and an alarm generation unit 190.

[0079] The alarm generating unit 190 generates an alarm when the number of passengers estimated by the first estimation unit 120 or the second estimation unit 140 is equal to or greater than a predetermined value. Here, the alarm is, for example, a sound generated from a speaker of the terminal 500 or a light generated from a lamp of the terminal 500, but is not limited to this example. For example, the alarm generating unit 190 may display information indicating that the number of passengers is equal to or greater than a predetermined value and information regarding the number of passengers on a display of the terminal 500. At this time, the alarm generating unit 190 may further display a captured image showing a vehicle in which the number of passengers is estimated to be equal to or greater than a predetermined value, a location where the vehicle was located, and information on the time when the vehicle was located. Note that the alarm generating unit 190 may generate an alarm on a speaker, lamp, or display connected to the information processing device 103 instead of the speaker, lamp, or display of the terminal 500 described above. In this way, the alarm generating unit 190 generates an alarm when the estimated number of passengers is equal to or greater than a predetermined value when the matching is true. The alarm generating unit 190 is an example of an alarm generating means.

[0080] The alarm generating unit 190 performs processing for generating an alarm, for example, after the processing in S104 or S106 in the flowchart of FIG.

[0081] In this way, the information processing device 103 of the fourth embodiment generates an alarm when the estimated number of passengers is equal to or greater than a predetermined value. A passenger capacity is set for each vehicle. The information processing device 103 can quickly find a vehicle with a number of passengers equal to or greater than the passenger capacity by, for example, setting a value equal to or greater than the vehicle's passenger capacity as the predetermined value.

[0082] Furthermore, for example, when the number of passengers is estimated based on information related to traffic signals, an abnormal number (e.g., 50) of passengers may be estimated from one vehicle. In this case, it is found that the vehicle has about 50 communication devices. The information processing device 103 of the fourth embodiment can also find such a suspicious vehicle having a large number of communication devices.

[0083] <Fifth embodiment> The information processing device according to the present disclosure may have the following configuration.

[0084] Fig. 10 is a block diagram showing an example of a functional configuration of the information processing device 104. The information processing system 1000 shown in Fig. 1 may be provided with the information processing device 104 instead of the information processing device 100. As shown in Fig. 10, the information processing device 104 includes a determination unit 111 and a first estimation unit 121.

[0085] The determination unit 111 determines whether or not the number of passengers in the vehicle can be estimated from a captured image of the vehicle. For example, the determination unit 111 acquires, via the Internet, image capture environment data that may affect image capture performed by the image capture device 200. Then, the determination unit 111 determines whether or not the number of passengers can be estimated based on, for example, the image capture environment data.

[0086] When it is determined that the number of passengers cannot be estimated, the first estimation unit 121 estimates the number of passengers based on information about signals detected from a range including the vehicle. For example, the first estimation unit 121 acquires information identifying a communication device extracted from detection data generated by the sensor 300. Then, the first estimation unit 121 estimates the number of passengers in the vehicle using, for example, the information identifying the communication device.

[0087] 11 is a flowchart for explaining an example of the operation of the information processing device 104. First, the determination unit 111 determines whether or not the number of passengers in the vehicle can be estimated from a captured image of the vehicle (S501). Next, when the determination unit 111 determines that the number of passengers cannot be estimated, the first estimation unit 121 estimates the number of passengers based on information about signals detected from a range including the vehicle (S502).

[0088] In this way, the information processing device 104 of the fifth embodiment determines whether or not the number of passengers in the vehicle can be estimated from a captured image of the vehicle, and if it is determined that the number of passengers cannot be estimated, estimates the number of passengers based on information about signals detected from a range including the vehicle. As a result, even if it is difficult to estimate the number of passengers in the vehicle from a captured image, the information processing device 104 can estimate the number of passengers from signals detected from a range including the vehicle. That is, the information processing device 104 of the fifth embodiment has the effect of being able to estimate the number of passengers in the vehicle even if it is difficult to estimate the number of passengers from a captured image.

[0089] <Example of hardware configuration of information processing device> The hardware constituting the information processing device of the above-mentioned first, second, third, fourth, and fifth embodiments will be described. Fig. 12 is a block diagram showing an example of the hardware configuration of a computer device that realizes the information processing device in each embodiment. Each block shown in Fig. 12 can be realized by a combination of a computer device 10 that realizes the information processing device and information processing method in each embodiment and software.

[0090] 12, a computer device 10 includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, a storage device 14, an input / output interface 15, a bus 16, and a drive device 17. The information processing device may be realized by a plurality of electric circuits.

[0091] The storage device 14 stores a program (computer program) 18. The processor 11 executes the program 18 of the information processing device using the RAM 12. Specifically, for example, the program 18 includes a program that causes a computer to execute the processes shown in FIG. 4, FIG. 6, FIG. 8, and FIG. 11. In response to the processor 11 executing the program 18, the functions of each component of the information processing device (the above-mentioned determination units 110, 111, the first estimation units 120, 121, the extraction unit 130, the second estimation unit 140, the registration unit 150, the control unit 160, the collation unit 170, the output unit 180, the alarm generation unit 190, etc.) are realized. The program 18 may be stored in the ROM 13. In addition, the program 18 may be recorded in the storage medium 20 and read out using the drive device 17, or may be transmitted to the computer device 10 from an external device (not shown) via a network (not shown).

[0092] The input / output interface 15 exchanges data with peripheral devices (such as a keyboard, a mouse, and a display device) 19. The input / output interface 15 functions as a means for acquiring or outputting data. The bus 16 connects the various components.

[0093] There are various variations in the method of implementing the information processing device. For example, the information processing device may be implemented as a dedicated device. Furthermore, the information processing device may be implemented based on a combination of multiple devices.

[0094] The scope of each embodiment also includes a processing method for recording a program for implementing each component of the function of each embodiment on a storage medium, reading the program recorded on the storage medium as code, and executing the program on a computer. That is, a computer-readable storage medium is also included in the scope of each embodiment. Also, the storage medium on which the above-mentioned program is recorded and the program itself are included in each embodiment.

[0095] The storage medium is, for example, but not limited to, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD (Compact Disc)-ROM, a magnetic tape, a non-volatile memory card, or a ROM. Furthermore, the programs recorded on the storage medium are not limited to programs that execute processes independently, but also include programs that operate on an OS (Operating System) and execute processes in cooperation with other software and functions of an expansion board, which are included in the scope of each embodiment.

[0096] Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above embodiment. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0097] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0098] <Additional Notes> [Appendix 1] A determination means for determining whether or not the number of passengers in the vehicle can be estimated from a photographed image of the vehicle; and a first estimation means for estimating the number of passengers based on information about a signal detected in a range including the vehicle when it is determined that the number of passengers cannot be estimated. Information processing device.

[0099] [Appendix 2] The determination means determines whether or not the number of passengers in the vehicle can be estimated from the captured image based on weather data for a location where an image capturing device that captures the vehicle is installed or information regarding an abnormality in the image capturing device. 2. An information processing device according to claim 1.

[0100] [Appendix 3] a second estimation means for estimating the number of passengers based on the captured image, The determination means determines whether or not the number of occupants of the vehicle can be estimated from the captured image, depending on the number of occupants estimated based on the captured image. 3. The information processing device according to claim 1 or 2.

[0101] [Appendix 4] the information regarding the signal is information for identifying a communication device that outputs the signal, an extraction means for extracting information for identifying the communication device from a signal detected in a range including the vehicle; 4. An information processing device according to any one of claims 1 to 3.

[0102] [Appendix 5] a control unit that controls a sensor to detect the signal from a range including the vehicle when it is determined that the number of passengers cannot be estimated; 5. An information processing device according to any one of claims 1 to 4.

[0103] [Appendix 6] a registration means for registering information about the signal, the estimated number of passengers, and the photographed image in a database in association with each other; 6. An information processing device according to any one of claims 1 to 5.

[0104] [Appendix 7] a comparison means for comparing information about a signal detected in a range including the vehicle with information about the signal registered in the database; 7. The information processing device according to claim 6.

[0105] [Appendix 8] The information regarding the signal is radio wave characteristic information indicating characteristics of a signal waveform of a radio wave output by a communication device. An information processing device according to any one of appendix 1 to 7.

[0106] [Appendix 9] The information regarding the signal is a MAC address included in the signal transmitted by the communication device. An information processing device according to any one of appendix 1 to 7.

[0107] [Appendix 10] an alarm generating means for generating an alarm when the estimated number of passengers is equal to or greater than a predetermined value; 10. An information processing device according to any one of claims 1 to 9.

[0108] [Appendix 11] an imaging device that captures an image of the vehicle and generates the captured image; a sensor for detecting the signal from a range including the vehicle; An information processing device according to any one of Supplementary Notes 1 to 10, Information processing system.

[0109] [Appendix 12] determining whether or not the number of passengers in the vehicle can be estimated from a photographed image of the vehicle; If it is determined that the number of passengers cannot be estimated, the number of passengers is estimated based on information about a signal detected in an area including the vehicle. Information processing methods.

[0110] [Appendix 13] determining whether or not the number of passengers in the vehicle can be estimated from the captured image based on weather data for a location where an image capturing device that captures the vehicle is installed or information regarding an abnormality in the image capturing device; 13. The information processing method according to claim 12.

[0111] [Appendix 14] Estimating the number of passengers based on the captured image; determining whether or not the number of occupants of the vehicle can be estimated from the photographed image according to the number of occupants estimated based on the photographed image; 14. The information processing method according to claim 12 or 13.

[0112] [Appendix 15] the information regarding the signal is information for identifying a communication device that outputs the signal, extracting information identifying the communication device from a signal detected in a range including the vehicle; 15. An information processing method according to any one of appendixes 12 to 14.

[0113] [Appendix 16] When it is determined that the number of passengers cannot be estimated, control is performed to cause a sensor to detect the signal from a range including the vehicle. 16. An information processing method according to any one of appendix 12 to 15.

[0114] [Appendix 17] registering the information about the signal, the estimated number of passengers, and the captured image in a database in association with each other; 17. An information processing method according to any one of appendix 12 to 16.

[0115] [Appendix 18] comparing information about signals detected in a range including the vehicle with information about the signals registered in the database; 18. An information processing method according to claim 17.

[0116] [Appendix 19] The information regarding the signal is radio wave characteristic information indicating characteristics of a signal waveform of a radio wave output by a communication device. 19. An information processing method according to any one of appendix 12 to 18.

[0117] [Appendix 20] The information regarding the signal is a MAC address included in the signal transmitted by the communication device. 19. An information processing method according to any one of appendix 12 to 18.

[0118] [Appendix 21] If the estimated number of passengers is equal to or greater than a predetermined value, an alarm is generated. 21. An information processing method according to any one of appendixes 12 to 20.

[0119] [Appendix 22] A process of determining whether or not the number of passengers in a vehicle can be estimated from a photographed image of the vehicle; and if it is determined that the number of passengers cannot be estimated, estimating the number of passengers based on information about signals detected in an area including the vehicle. A computer-readable storage medium.

[0120] [Appendix 23] A program for causing a computer to execute a process of determining whether or not the number of passengers in the vehicle can be estimated from the captured image based on weather data for a location where an image capturing device that captures the vehicle is installed or information regarding an abnormality in the image capturing device. 23. The computer-readable storage medium of claim 22.

[0121] [Appendix 24] A process of estimating the number of passengers based on the captured image; and determining whether or not the number of occupants of the vehicle can be estimated from the photographed image, depending on the number of occupants estimated based on the photographed image. 24. The computer-readable storage medium of claim 22 or 23.

[0122] [Appendix 25] the information regarding the signal is information for identifying a communication device that outputs the signal, A program for causing a computer to execute a process of extracting information for identifying the communication device from a signal detected in a range including the vehicle. 25. A computer-readable storage medium according to any one of claims 22 to 24.

[0123] [Appendix 26] a program for causing a computer to execute a process of controlling a sensor to detect the signal from a range including the vehicle when it is determined that the number of passengers cannot be estimated; 26. A computer-readable storage medium according to any one of claims 22 to 25.

[0124] [Appendix 27] A program for causing a computer to execute a process of correlating information about the signal, the estimated number of passengers, and the photographed image and registering them in a database. 27. A computer-readable storage medium according to any one of claims 22 to 26.

[0125] [Appendix 28] A program for causing a computer to execute a process of comparing information about a signal detected in a range including the vehicle with information about the signal registered in the database. 28. The computer-readable storage medium of claim 27.

[0126] [Appendix 29] The information regarding the signal is radio wave characteristic information indicating characteristics of a signal waveform of a radio wave output by a communication device. 29. A computer-readable storage medium according to any one of claims 22 to 28.

[0127] [Appendix 30] The information regarding the signal is a MAC address included in the signal transmitted by the communication device. 29. A computer-readable storage medium according to any one of claims 22 to 28.

[0128] [Appendix 31] a program for causing a computer to execute a process for generating an alarm when the estimated number of passengers is equal to or greater than a predetermined value; 31. A computer-readable storage medium according to any one of claims 22 to 30. [Explanation of symbols]

[0129] 100, 101, 102, 103, 104 Information processing device 110, 111 Judgment section 120, 121 1st estimation part 130 Extraction part 140 Second estimation part 150 Registration Department 160 Control section 170 Collation Unit 180 Output section 190 Alarm generating section 200 Imaging Device 300 Sensors 400 storage device 500 terminals

Claims

1. A determination means for determining whether or not the number of passengers in the vehicle can be estimated from a photographed image of the vehicle; and a first estimation means for estimating the number of passengers based on information about a signal detected in an area including the vehicle when the determination means determines that the number of passengers cannot be estimated. Information processing device.

2. The determination means determines whether or not the number of passengers in the vehicle can be estimated from the captured image based on weather data of a location where an image capturing device that captures the vehicle is installed or information regarding an abnormality in the image capturing device. The information processing device according to claim 1 .

3. a second estimation means for estimating the number of passengers based on the captured image, The determination means determines whether or not the number of occupants of the vehicle can be estimated from the captured image, depending on the number of occupants estimated based on the captured image.

3. The information processing device according to claim 1 or 2.

4. the information regarding the signal is information for identifying a communication device that outputs the signal, an extraction means for extracting information for identifying the communication device from a signal detected in a range including the vehicle; 4. The information processing apparatus according to claim 1.

5. a control unit that controls a sensor to detect the signal from a range including the vehicle when it is determined that the number of passengers cannot be estimated; 5. The information processing device according to claim 1.

6. a registration means for registering information about the signal, the estimated number of passengers, and the photographed image in a database in association with each other; 6. An information processing apparatus according to claim 1.

7. a comparison means for comparing information about a signal detected in a range including the vehicle with information about the signal registered in the database; The information processing device according to claim 6.

8. The information regarding the signal is radio wave characteristic information indicating characteristics of a signal waveform of a radio wave output by a communication device.

8. An information processing apparatus according to claim 1.

9. determining whether or not the number of passengers in the vehicle can be estimated from a photographed image of the vehicle; If it is determined that the number of passengers cannot be estimated, the number of passengers is estimated based on information related to a signal detected from an area including the vehicle. Information processing methods.

10. A process of determining whether or not the number of passengers in a vehicle can be estimated from a photographed image of the vehicle; A program that causes a computer to execute a process of estimating the number of passengers based on information regarding signals detected from an area including the vehicle, if the determination process determines that the number of passengers cannot be estimated.

Images