Information processing device, information processing method, and recording medium

Abstract

An information processing device according to the present invention comprises a generation unit, a setting unit, and a detection unit. The generation unit generates planar layout information using original information obtained from at least one of GIS information, an image captured by an onboard camera, and point group information generated using an onboard LiDAR sensor, the planar layout information representing a plan view of the layout of a lot region that includes a dwelling and a parking site at which an automobile parks. The setting unit sets a monitoring area on the basis of a designation made to the planar layout information. The detection unit uses analysis results for sensor information generated at an onboard sensor to detect predetermined suspicious events regarding moving bodies in the monitoring area.

Description

Information processing device, information processing method, and recording medium

[0001] The present invention relates to an information processing device, an information processing method, and a recording medium.

[0002] For example, Patent Document 1 describes a vehicle security device. This vehicle security device includes a camera, a positioning device, and a control device. The camera is mounted on a vehicle and captures images of the interior and exterior of the vehicle while the vehicle is parked. The positioning device is mounted on the vehicle and determines the vehicle's parking position. The control device issues an alarm when it detects a suspicious person in the image captured by the camera, and changes the detection range for the suspicious person depending on the parking position.

[0003] Patent Document 1 describes that by expanding the detection range in a home parking lot compared to a general parking lot other than the home, it is possible to improve not only the crime prevention performance of the vehicle but also the crime prevention performance of the home. Patent Document 1 also describes that if the resolution of the video captured at home is set higher than when the car is parked outside the home, the movements and facial expressions of a suspicious person can be captured in more detail, thereby improving the accuracy of detecting a suspicious person.

[0004] Japanese Patent Application Laid-Open No. 2020-149088

[0005] However, even if the detection range is expanded or the resolution is set high, it is not always possible to sufficiently monitor an appropriate area to improve safety in a home, such as for home security. In general, there is often a trade-off between detection range and resolution in image capture devices. Therefore, for example, monitoring a wide area, including an area where monitoring is not necessary, may result in a decrease in image quality and a decrease in monitoring accuracy. Furthermore, for example, when monitoring is required beyond the detection range of the image capture device, monitoring using a sensor other than the image capture device installed in the vehicle may improve safety in the home.

[0006] One of the objectives of the present disclosure is to improve safety in homes by using vehicle-mounted sensors.

[0007] The information processing device of the present disclosure includes: a planar layout generation means for generating planar layout information showing in plan view the layout of a site area including a parking lot where automobiles are parked and a residence, using original information obtained from at least one of GIS information, images captured by an on-board camera, and point cloud information generated using an on-board LiDAR sensor; a setting means for setting a monitoring area based on specifications for the planar layout information; and a detection means for detecting predetermined suspicious events related to moving objects present in the monitoring area, using analysis results of sensor information generated by the on-board sensor.

[0008] The information processing method disclosed herein involves one or more computers generating planar layout information showing in plan view the layout of a site area including a parking lot where automobiles are parked and a residence, using original information obtained from at least one of GIS information, images captured by an on-board camera, and point cloud information generated using an on-board LiDAR sensor, setting a monitoring area based on specifications for the planar layout information, and detecting predetermined suspicious events related to moving objects present in the monitoring area using the analysis results of sensor information generated by the on-board sensor.

[0009] The recording medium of the present disclosure has recorded thereon a program for causing one or more computers to generate planar layout information showing in plan view the layout of a site area including a parking lot where automobiles are parked and a residence, using original information obtained from at least one of GIS information, images captured by an on-board camera, and point cloud information generated using an on-board LiDAR sensor; set a monitoring area based on specifications for the planar layout information; and detect predetermined suspicious events related to moving objects present in the monitoring area using analysis results of sensor information generated by the on-board sensor.

[0010] According to the present disclosure, it is possible to improve safety in homes by using vehicle-mounted sensors.

[0011] 1 is a block diagram showing a configuration example of a first information processing device according to the present disclosure. FIG. 2 is a flowchart showing an operation example of the first information processing device according to the present disclosure. FIG. 3 is a block diagram showing a configuration example of a first information processing system according to the present disclosure. FIG. 4 is a diagram showing an example of a premises area and a monitoring area according to the present disclosure. FIG. 5 is a block diagram showing a configuration example of a first in-vehicle device according to the present disclosure. FIG. 6 is a flowchart showing an operation example of the first in-vehicle device according to the present disclosure. FIG. 7 is a block diagram showing a detailed configuration example of the first information processing device according to the present disclosure. FIG. 8 is a flowchart showing an operation example of the first information processing device according to the present disclosure. FIG. 9 is a block diagram showing a configuration example of a first setting unit according to the present disclosure. FIG. 10 is a flowchart showing an operation example of the first setting unit according to the present disclosure. FIG. 11 is a diagram showing a physical configuration example of the first information processing device according to the present disclosure. FIG. 12 is a block diagram showing a configuration example of a second information processing system according to the present disclosure. FIG. 13 is a block diagram showing a configuration example of a second information processing device according to the present disclosure. FIG. 14 is a flowchart showing an operation example of the second information processing device according to the present disclosure. FIG. 15 is a block diagram showing a configuration example of a third information processing device according to the present disclosure. FIG. 16 is a flowchart showing an operation example of a fourth information processing device according to the present disclosure. FIG. 17 is a block diagram showing a configuration example of a third information processing system according to the present disclosure.

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings, similar components are designated by similar reference numerals, and descriptions thereof will be omitted as appropriate. In addition, in this disclosure, the drawings relate to one or more embodiments.

[0013] First Embodiment (Configuration Example of Information Processing Apparatus 100) As shown in FIG. 1, the information processing apparatus 100 includes a generating unit 110, a setting unit 140, and a detecting unit 150.

[0014] The generation unit 110 generates planar layout information showing in plan view the layout of a site area including a parking lot where automobiles are parked and a house, using original information obtained from at least one of GIS information, images taken by an on-board camera, and point cloud information generated using an on-board LiDAR sensor.

[0015] The setting unit 140 sets a monitoring area based on the designation for the planar layout information.

[0016] The detection unit 150 uses the analysis results of sensor information generated by the vehicle-mounted sensor to detect a predetermined suspicious event related to a moving object present in a monitoring area.

[0017] According to the information processing device 100, a monitoring area is set using the layout of the site area, so that an on-board sensor can be used to monitor an appropriate monitoring area to ensure the safety of the home, thereby improving the safety of the home using an on-board sensor.

[0018] (Example of Operation of Information Processing Device 100) The information processing device 100 executes information processing as shown in FIG.

[0019] The generation unit 110 generates planar layout information showing in plan view the layout of the site area including the parking lot where the car is parked and the house, using original information obtained from at least one of GIS information, images taken by an on-board camera, and point cloud information generated using an on-board LiDAR sensor (step S110).

[0020] The setting unit 140 sets a monitoring area based on the designation for the planar layout information (step S140).

[0021] The detection unit 150 uses the analysis results of the sensor information generated by the on-board sensor to detect a predetermined suspicious event related to a moving object present in the monitoring area (step S150).

[0022] According to this information processing, the layout of the site area is used to set the monitoring area, so that the on-board sensor can be used to monitor the appropriate monitoring area to ensure the safety of the house, thereby improving the safety of the house using the on-board sensor.

[0023] (Detailed Example) Hereinafter, a detailed example of the information processing device 100 and its operation will be described.

[0024] The information processing device 100 is a device for monitoring a monitoring area related to a site area using an on-board sensor.

[0025] The site area is an area including a site, as shown in an example in Fig. 3. The site includes a parking lot where the automobile 10 equipped with the on-board sensor is parked, and a residence. The site may further include a garden, etc.

[0026] The site area may further include the periphery of the site, which is, for example, an area within a predetermined distance (for example, 2 meters) from the outer edge of the site, including, for example, roads, sites, etc. adjacent to the site.

[0027] (Regarding the information processing system S1) The information processing system S1 is a system for monitoring a monitoring area related to a site area using on-board sensors. As shown in FIG. 4, for example, the information processing system S1 includes a plurality of on-board sensors 20 and on-board devices 30 mounted on an automobile 10, and an information processing device 100. Note that the number of on-board sensors 20 may be one. The automobile 10 is, for example, an automobile parked in a parking lot included in the site area.

[0028] The multiple on-board sensors 20 and the on-board device 30 are connected to each other by wires, for example, inside the automobile 10. As a result, the multiple on-board sensors 20 and the on-board device 30 are connected so as to be able to transmit and receive various information, such as sensor information, to and from each other.

[0029] The in-vehicle device 30 and the information processing device 100 are connected to each other via, for example, a communication network NT. The communication network NT may be configured as, for example, a wired network, a wireless network, or a combination of these. This allows the in-vehicle device 30 and the information processing device 100 to be connected to each other so that, for example, they can transmit and receive various information to and from each other.

[0030] It should be noted that the multiple on-board sensors 20, on-board devices 30, and information processing device 100 only need to be connected so as to be able to transmit and receive information to and from each other, and the configuration for connecting them is not limited to the example described above.

[0031] (Regarding the on-board sensors 20) Each of the multiple on-board sensors 20 is a sensor for detecting objects inside or around the automobile 10. Each of the multiple on-board sensors 20 generates sensor information indicating objects inside or around the automobile 10. By analyzing this sensor information, it is possible to detect objects inside or around the automobile 10.

[0032] Here, "object" includes people and things, and the same applies hereinafter.

[0033] The multiple on-board sensors 20 include at least one of an on-board camera, an on-board LiDAR (Light Detection and Ranging) sensor, and an on-board object presence detection sensor.

[0034] The on-board camera is, for example, a camera mounted on the automobile 10 to capture images of the exterior or interior of the automobile 10. The on-board camera captures, for example, an image of a predetermined capture area outside or inside the automobile 10 and generates the captured image.

[0035] The on-board LiDAR sensor is a sensor used for LiDAR. LiDAR is a technology that irradiates a laser beam and uses the reflected light to measure the distance to an object, the shape of the object, and the like. The on-board LiDAR sensor may include, for example, a laser irradiator that irradiates the laser beam and an optical receiver that receives the reflected light. The on-board LiDAR sensor, for example, irradiates a laser beam onto a predetermined area outside the automobile 10 and generates information indicative of the reflected light. This information indicative of the reflected light is LiDAR information generated by the on-board LiDAR sensor.

[0036] The ultrasonic sensor, the infrared sensor, and the millimeter wave sensor are each examples of an object presence detection sensor for detecting the presence or absence of an object in the vicinity of the automobile 10 .

[0037] The object presence detection sensor is used to prevent collisions of the automobile 10, and detects objects that exist within a predetermined distance from the automobile 10. The object presence detection sensor, for example, irradiates sound waves, electromagnetic waves, etc. to the outside of the automobile 10 and generates object reflection information that indicates the reflected waves.

[0038] The object presence detection sensor may be, for example, at least one of an ultrasonic sensor, an infrared sensor, and a millimeter wave sensor. In detail, each of the ultrasonic sensor, infrared sensor, and millimeter wave sensor may include, for example, an oscillator that emits ultrasonic waves, infrared rays, or millimeter waves, respectively, and a receiver that receives the reflected waves.

[0039] (Regarding the on-vehicle device 30) When the on-vehicle device 30 acquires sensor information from the on-vehicle sensors 20, the on-vehicle device 30 performs analysis using the sensor information and transmits the analysis result to the information processing device 100. In detail, for example, when the on-vehicle device 30 acquires sensor information from each of the on-vehicle sensors 20, the on-vehicle device 30 analyzes the sensor information and transmits the analysis result to the information processing device 100.

[0040] As shown in FIG. 5 , the in-vehicle device 30 includes a sensor information acquisition unit 31 and an analysis unit 32 .

[0041] The sensor information acquisition unit 31 acquires the sensor information generated by each of the multiple on-board sensors 20 .

[0042] The analysis unit 32 analyzes the sensor information acquired by the sensor information acquisition unit 31 and generates an analysis result.

[0043] The in-vehicle device 30 executes an analysis process such as that shown in FIG.

[0044] The sensor information acquisition unit 31 acquires the sensor information generated by each of the multiple on-board sensors 20 (step S31).

[0045] The analysis unit 32 analyzes the sensor information acquired in step S31 and generates an analysis result (step S32).

[0046] (Regarding the sensor information acquisition unit 31) The sensor information acquisition unit 31 may acquire sensor information from each of the in-vehicle sensors 20, for example, based on a user instruction or while monitoring the monitoring area. Note that triggers for the sensor information acquisition unit 31 to acquire sensor information are not limited to these.

[0047] (About the analysis unit 32) The analysis unit 32 is a device that generates an analysis result using sensor information. The analysis unit 32 has an analysis function for performing an analysis process on the sensor information. For example, the analysis unit 32 detects objects outside or inside the automobile 10 by analyzing the sensor information, and generates mobile object information related to a mobile object among the objects as an analysis result. The analysis result (mobile object information) may include, for example, one or more of the time the mobile object was detected, the position of the mobile object, the moving speed of the mobile object, the moving path of the mobile object, the attitude of the mobile object, and attributes of the mobile object.

[0048] The attributes of the moving object may include, for example, the type of the moving object, such as a person, a car, etc. If the moving object is a person, the attributes of the moving object may include, for example, one or more of height, build, clothing, gender, age group, etc. If the moving object is a car, the attributes of the moving object may include, for example, one or more of model, size, color, etc.

[0049] A detailed example of a function for generating such an analysis result will be described below. Note that the information included in the analysis result is not limited to the above example.

[0050] The analysis unit 32 may have an analysis function according to the type of sensor information, such as the above-mentioned image, LiDAR information, object reflection information, etc., and corresponds to the type of the on-board sensor 20.

[0051] In more detail, for example, the analysis unit 32 may analyze the sensor information generated by the multiple on-board sensors 20 using an analysis function corresponding to the type of the sensor information. Then, the analysis unit 32 may integrate the individual analysis results obtained by analyzing each piece of sensor information to generate an analysis result (overall analysis result) obtained by analyzing all of the sensor information generated by the multiple on-board sensors 20. When simply referred to as an "analysis result," this means the overall analysis result.

[0052] When integrating individual analysis results, for example, objects detected from each sensor information (i.e., objects included in the individual analysis results) may be treated as individual objects and combined to generate an analysis result. Furthermore, when integrating individual analysis results, if objects included in the individual analysis results are within a predetermined distance, such nearby objects may be treated as a single object to generate an analysis result. In this case, the position of the nearby object, the time at which the object was detected, etc. may be, for example, an average of the positions, times, etc. included in the individual analysis results, or any one of the positions, times, etc. included in the individual analysis results.

[0053] The method for integrating the individual analysis results is not limited to the method described here.

[0054] In general, the accuracy of the individual analysis results varies depending on the environment, such as the brightness around the automobile 10. Therefore, by integrating the individual analysis results, it is possible to obtain more accurate analysis results.

[0055] Below, an example of an analysis function according to the type of sensor information, that is, an example of a method for generating individual analysis results, will be described.

[0056] (Example of Image Analysis Function) For example, the analysis unit 32 has one or more analysis functions that perform processing (analysis processing) for analyzing images such as videos and still images. Each of the analysis functions performs analysis by extracting image feature vectors using, for example, a machine learning model that has learned to perform analysis according to the function. Note that, although an example of analyzing images using a machine learning model will be described here, image analysis is not limited to this, and general image processing techniques such as pattern matching may also be used.

[0057] The analysis functions provided by the analysis unit 32 include, for example, one or more of: (1) object detection function, (2) face analysis function, (3) human figure analysis function, (4) posture analysis function, (5) behavior analysis function, (6) appearance attribute analysis function, (7) gradient feature analysis function, (8) color feature analysis function, and (9) movement line analysis function.

[0058] (1) The object detection function detects an object from an image. The object detection function can also determine the position of an object within an image. For example, YOLO (You Only Look Once) is a model that can be applied to the object detection process.

[0059] (2) The face analysis function detects human faces from images, extracts feature vectors of the detected faces, and classifies the detected faces. The face analysis function can also determine the position of the face in the image, as well as the direction of the face and gaze. The face analysis function can also determine the identity of people detected from different images based on the similarity between the facial features of people detected from different images.

[0060] (3) The human morphology analysis function extracts the physical characteristics of people included in an image (for example, values ​​indicating overall characteristics such as whether they are fat or thin, their height, and their clothing), and classifies (classifies) people included in an image. The human morphology analysis function can also identify the position of a person within an image. The human morphology analysis function can also determine the identity of people included in different images based on the physical characteristics of the people included in different images.

[0061] (4) The posture analysis function detects the joint points of a person from an image and creates a stick figure model by connecting the joint points. The posture analysis function then uses the information from the stick figure model to estimate the posture of the person, extract a feature vector of the estimated posture (posture feature vector), and classify (classify) the people included in the image. The posture analysis function can also determine the identity of people included in different images based on the posture feature vectors of the people included in different images.

[0062] For example, the posture analysis function estimates a person's posture, such as standing, crouching, or bending, from an image and extracts a posture feature vector indicating the posture of each person.Furthermore, for example, the posture analysis function can estimate the posture of an object detected using an object detection function or the like from an image and extract a posture feature vector indicating that posture.

[0063] (5) The behavior analysis function can estimate a person's movements using information about the stick figure model, changes in posture, etc., extract feature vectors of the person's movements (movement feature vectors), and classify (classify) people included in the image. The behavior analysis function can also estimate a person's height and identify the person's position in the image using information about the stick figure model. For example, the behavior analysis function can estimate a person's movements, such as changes or transitions in posture and movement (changes or transitions in position), from the image and extract movement feature vectors related to the person's movements.

[0064] (6) The appearance attribute analysis function can recognize appearance attributes associated with a person. The appearance attribute analysis function extracts feature vectors (appearance attribute feature vectors) related to the recognized appearance attributes and classifies (classifies) people included in an image. Appearance attributes are attributes of a person's appearance. Appearance attributes include, for example, one or more of age group, gender, type and color of clothing, type and color of shoes, hairstyle, whether or not a hat is worn, whether or not a tie is worn, whether or not glasses are worn, whether or not an umbrella is carried, and whether or not an umbrella is being used.

[0065] (7) The gradient feature analysis function extracts gradient feature quantities (gradient feature quantities) in an image. For gradient feature detection processing, techniques such as SIFT, SURF, RIFF, ORB, BRISK, CARD, and HOG can be applied.

[0066] SIFT is an abbreviation for Scale-Invariant Feature Transform. SURF is an abbreviation for Speeded-Up Robust Features. RIFF is an abbreviation for Rotation-Invariant Fast Feature. BRIEF is an abbreviation for Binary Robust Independent Elementary Features. ORB is an abbreviation for Oriented FAST and Rotated BRIEF. BRISK is an abbreviation for Binary Robust Invariant Scalable Keypoints. CARD is an abbreviation for Compact And Real-time Descriptors. HOG is an abbreviation for Histograms of Oriented Gradients.

[0067] (8) The color feature analysis function can detect an object from an image, extract a color feature vector of the detected object, and classify (classify) the detected object.

[0068] The color feature amount is, for example, a color histogram. The color feature analysis function can, for example, detect people and objects included in an image. Furthermore, for example, the color feature analysis function can classify each of the people and objects into classes.

[0069] (9) The flow line analysis function can determine the flow line (trajectory of movement) of a person included in a video, for example, using the results of the identity determination in any of the analysis functions (2) to (6) described above. In more detail, for example, by connecting people determined to be the same across different images in a time series, the flow line of that person can be determined. Note that, in cases where video footage is acquired using multiple in-vehicle cameras or the like that capture different shooting areas, the flow line analysis function can also determine the flow line across multiple videos captured in different shooting areas.

[0070] The image feature vector includes, for example, the object detection result of the object detection function, a face feature vector, a human body feature vector, a posture feature vector, a movement feature vector, an appearance attribute feature vector, a gradient feature vector, a color feature vector, and a movement line.

[0071] Each of the analysis functions (1) to (9) may appropriately use the results of analysis performed by other analysis functions.

[0072] (Example of LiDAR Information Analysis Function) The analysis unit 32 generates point cloud information using information generated by the on-board LiDAR sensor. A general technique may be used to generate the point cloud information. The analysis unit 32 uses the point cloud information to generate individual analysis information including at least one of the object's position, shape, detection time, etc.

[0073] (Example of Object Reflection Information Analysis Function) The analysis unit 32 uses the object reflection information generated by the object presence / absence detection sensor to generate individual analysis information including at least one of the object position, the time of detection, etc. A general technology may be used as a technology for generating such individual analysis information.

[0074] (Example of configuration of information processing device 100) As shown in FIG. 7, for example, the information processing device 100 includes a generation unit 110, a display control unit 120, a display unit 130, a setting unit 140, a detection unit 150, an alarm control unit 160, and a suspicious event storage unit 170.

[0075] The overview of the generating unit 110, the setting unit 140, and the detecting unit 150 is as described above.

[0076] The display control unit 120 causes the display unit 130 to display the planar layout information.

[0077] The notification control unit 160 issues a predetermined notification when a suspicious event is detected.

[0078] The suspicious event storage unit 170 stores a suspicious event log, which is a history of detected suspicious events.

[0079] The information processing device 100 executes information processing such as that shown in FIG.

[0080] The outline of steps S110, S140 and S150 is as described above.

[0081] The display control unit 120 causes the display unit 130 to display the planar layout information (step S120).

[0082] When a suspicious event is detected, the notification control unit 160 issues a predetermined notification (step S160).

[0083] The suspicious event storage unit 170 stores a suspicious event log, which is a history of detected suspicious events (step S170).

[0084] (Generation Unit 110) As described above, generation unit 110 generates planar layout information using original information.

[0085] The original information is information obtained from at least one of GIS (Geographic Information System) information, images captured by an on-board camera, and point cloud information. The original information may include, for example, at least one of GIS information, images, point cloud information, and information obtained by integrating the results of individual analysis of the images and point cloud information by the analysis unit 32. The on-board camera may be, for example, a camera mounted on the automobile 10. The point cloud information may be, for example, point cloud information generated by the analysis unit 32 using an on-board LiDAR sensor mounted on the automobile 10.

[0086] The planar layout information is information that shows the layout of the site area in a plan view (i.e., a view of the site area from above). The layout may include the arrangement of one or more predetermined locations, such as parking lots, houses, entrances to houses such as front doors and back doors, windows, fences, gates, and surrounding roads.

[0087] For example, when information obtained from multiple pieces of information among GIS information, images, and point cloud information is used as original information, the generator 110 may generate planar layout information by adjusting the layout of the site area indicated by the multiple pieces of information. In detail, for example, the planar layout information may be generated by adjusting the layout of the site area indicated by each of the multiple pieces of information among GIS information, images, and point cloud information so that the positions of predetermined locations match.

[0088] (Display Control Unit 120 and Display Unit 130) As described above, the display control unit 120 may cause the display unit 130 to display, for example, planar layout information.

[0089] The display unit on which the display control unit 120 displays information is not limited to the display unit 130 included in the information processing device 100, but may be included in, for example, a terminal device (not shown) connected via the communication network NT. The terminal device is, for example, a tablet terminal or a smartphone used by a user such as a resident.

[0090] (Regarding the setting unit 140) The setting unit 140 sets a monitoring area based on specifications for the planar layout information generated by the generation unit 110. The monitoring area to be set is, for example, the hatched area in Fig. 4. The figure shows an example of a monitoring area that includes part of a parking lot and a path from a gate to the entrance. Note that the monitoring area shown in Fig. 4 is only an example, and the monitoring area to be set is not limited to this.

[0091] In detail, for example, the setting unit 140 includes an automatic generation unit 141 and a monitoring area setting unit 142 as shown in FIG.

[0092] The automatic generation unit 141 generates an automatically generated area for the site area.

[0093] The monitoring area setting unit 142 sets a monitoring area based on the automatically generated area and specifications for the planar layout information.

[0094] More specifically, for example, the setting unit 140 executes a setting process (step S140) as shown in FIG.

[0095] The automatic generation unit 141 generates an automatically generated area for the site area (step S141).

[0096] The monitoring area setting unit 142 sets a monitoring area based on the automatically generated area and the designation for the planar layout information (step S142).

[0097] (Regarding the Automatic Generation Unit 141) The automatic generation unit 141 automatically generates an automatically generated area, for example, for a site area, in accordance with predetermined automatic generation conditions. The automatic generation conditions include, for example, an area within a predetermined range from predetermined recommended monitoring locations, a route connecting the recommended monitoring locations, an area along the route, etc.

[0098] The recommended monitoring locations are locations where it is possible to enter the premises from outside, or to look into the premises or the inside of the house from outside, etc. In more detail, for example, the recommended monitoring locations are at least one of the entrance, gate, window, back door, etc. Note that the recommended monitoring locations are not limited to those exemplified here.

[0099] (Monitoring Area Setting Unit 142) The monitoring area setting unit 142 sets a monitoring area based on the automatically generated area generated by the automatic generation unit 141 and the designation for the planar layout information displayed on the display unit 130.

[0100] At this time, the user may designate an area to be monitored (designated area) by referring to the planar layout information displayed on the display unit 130. The display unit 130 may display an automatically generated area together with the planar layout information.

[0101] As described above, the planar layout information may be displayed on a terminal device (not shown) instead of the display unit 130. In this case, the designation for setting the monitoring area may be performed using the terminal device.

[0102] The monitoring area setting unit 142 may set the entire area including the automatically generated area and the specified area (specified area) as the monitoring area. This makes it possible to set the entire area that is generally likely to pose a threat to the safety of the home and the area that the user considers to pose a high risk to the safety of the home as the monitoring area.

[0103] The monitoring area setting unit 142 may set only the designated area as the monitoring area. This allows the user to set only the areas that the user considers to be highly likely to threaten the safety of the home as the monitoring area. The designated area may be specified with reference to the automatically generated area, as described above. This allows the monitoring area to be set by excluding the automatically generated area that is less likely to need monitoring depending on the individual circumstances of the home.

[0104] (Regarding the detection unit 150) The detection unit 150 detects a predetermined suspicious event related to a moving object present in a monitoring area by using an analysis result of sensor information generated by the on-board sensor 20 mounted on the automobile 10. The analysis result is generated by, for example, the analysis unit 32.

[0105] The moving body may be any object other than a fixed object such as a gate, and is not limited to moving objects but also includes stationary objects.

[0106] A suspicious event may be predefined in relation to one or more suspicious attributes. The suspicious attributes are attributes of elements (suspicious elements) that constitute the suspicious event. The suspicious attributes may include, for example, (A) the behavior of the moving object, (B) the time period in which the moving object was present, (C) the attributes of the moving object, and (D) the surrounding circumstances.

[0107] (A) The movement of a moving object may include, in detail, one or more movement attributes such as (A-1) movement speed, (A-2) position, (A-3) behavior, (A-4) movement path, and (A-5) frequency.

[0108] (A-1) The suspicious element related to the moving speed is, for example, at least one of stopping, slow speed below a predetermined speed, and the like.

[0109] (A-2) The suspicious element belonging to a location is, for example, at least one of being within a predetermined distance from a reference location, etc. The reference location is, for example, a location that is likely to be connected to danger in a house, such as a gate, a window, a back door, a fence, etc.

[0110] (A-3) Suspicious elements of behavior include, for example, at least one of the following: a crouching or half-squatting posture; a face or gaze directed toward the residence, on the premises, or indoors; not pressing the intercom; and no contact with residents.

[0111] (A-4) The suspicious element belonging to the movement route is, for example, at least one of the movement trajectory and the like.

[0112] (A-5) The suspicious element belonging to frequency is, for example, at least one of the frequency of a predetermined action within a predetermined period of time.

[0113] The predetermined action here may be defined using one or more of the suspicious elements exemplified in (A-1) to (A-4), for example. The predetermined period may be determined according to the period during which the associated predetermined action is generally repeated in suspicious actions.

[0114] (B) The suspicious element belonging to the time period in which the moving object was present is, for example, at least one of nighttime, early morning, daytime, etc. Nighttime is, for example, 10 PM to 12 PM and 12 PM to 3 AM. Early morning is, for example, 3 AM to 5 AM. Daytime is, for example, a time period when residents are often not at home (for example, 10 AM to 3 PM). Note that the examples of time periods are not limited to those given here.

[0115] (C) The suspicious element belonging to the attributes of the moving body is, for example, at least one of being a person, being a person whose height is greater than a predetermined value, being a car, being on a motorbike or bicycle, and the like.

[0116] (D) The suspicious element belonging to the surrounding circumstances is, for example, at least one of the following: the brightness of the site area is dark and below a threshold value; the weather is cloudy or rainy; and the like.

[0117] Using these suspicious elements, suspicious events such as loitering, looking into something, suspicious posture, and suspicious movement can be defined as follows.

[0118] Prowling may be defined as, for example, moving slowly for a predetermined period of time or more, moving slowly a predetermined number of times or more, moving slowly a predetermined number of times or more within a predetermined period of time, etc. Prowling may also be defined as, for example, approaching a residence more frequently than a predetermined number of times without contacting the resident or without pressing the intercom.

[0119] Peering may be defined as, for example, walking or walking slowly while looking at a house within a predetermined distance from the property, or turning one's face into the house from a window or the like within a predetermined distance.

[0120] A suspicious posture may be defined as, for example, crouching, crouching under a vehicle, etc.

[0121] Suspicious movement may be defined as, for example, movement from the gate to the front door via a route that is not the shortest route from the gate to the front door.

[0122] (Regarding Resident Behavior Patterns) The detection unit 150 may detect suspicious events using the analysis results and resident behavior patterns of the residence. The resident behavior patterns are behavior patterns of the residents of the residences included in the site area. The resident behavior patterns may be set by the residents or the like and stored by the detection unit 150, for example.

[0123] The resident behavior pattern may include at least one of a route pattern related to the resident's travel route and a time pattern related to the resident's activity time. That is, the resident behavior pattern may include a route-time pattern including both a route pattern and a time pattern. The route-time pattern is a resident behavior pattern related to both the resident's travel route and activity time.

[0124] Suspicious events may be defined using resident behavior patterns.

[0125] For example, when a resident behavior pattern includes a route pattern, a suspicious event may include a mobile object moving along a route that differs from the route pattern. For example, when a resident behavior pattern includes a time pattern, a suspicious event may include a mobile object moving during a time period that differs from the time pattern. For example, a resident's route of travel may differ depending on the time period. Therefore, when a resident behavior pattern includes a route time pattern, a suspicious event may include a mobile object moving at a certain time period along a route that differs from the route pattern in the time pattern to which the time period belongs.

[0126] The time when the detection unit 150 detects a suspicious event (i.e., monitors the monitoring area) may be determined using a resident behavior pattern. For example, when the resident behavior pattern includes a time pattern, the detection unit 150 may detect a suspicious event using an analysis result of sensor information obtained at a monitoring time set using the time pattern. For example, the detection unit 150 may identify the monitoring time by statistically processing the time pattern, and store and set the identified monitoring time.

[0127] In this case, the monitoring period may be, for example, at least one of a period when the resident is absent from the house, a period when only children or elderly people are at home, and the like.

[0128] In more detail, for example, a period when the resident is absent from the residence is a period when the probability that the resident will be absent from the residence is equal to or greater than a predetermined value, etc. For example, a period when the only residents at home are children or elderly people (those under a predetermined age or above a predetermined age) is a period when the probability that the only residents at home are children or elderly people is equal to or greater than a predetermined value, etc.

[0129] Suspicious events may include exceptions such as excluding occupants of a home or excluding certain resident behavior patterns.

[0130] In more detail, for example, the detection unit 150 may store preset resident information and perform exception processing using the resident information to detect suspicious events. The resident information is information about the resident of the house, and may include at least one of face, age, height, physique, etc.

[0131] Furthermore, for example, when performing exception processing using resident behavior patterns, the detection unit 150 may include exception processing such as excluding route patterns of specific residents, such as children or elderly residents. This makes it possible to exclude events related to moving objects that travel along route patterns rarely used by specific residents from targets for detection as suspicious events. This reduces the possibility that events that are unlikely to be suspicious will be detected as suspicious events.

[0132] Furthermore, for example, when performing exception processing using resident behavior patterns, the detection unit 150 may include exception processing such as excluding route patterns of specific residents, such as children or elderly people, during specific time periods. This makes it possible to exclude events related to moving objects that travel along route patterns used by specific residents during specific time periods from targets for detection as suspicious events. This reduces the possibility that events that are unlikely to be suspicious will be detected as suspicious events.

[0133] (Regarding the notification control unit 160) The notification performed by the notification control unit 160 may be, for example, turning on a light mounted on the automobile 10 or sounding a horn mounted on the automobile 10. The notification may be, for example, turning on a light installed around the premises or sounding a buzzer installed around the premises. This makes it possible to issue a warning to a moving object in which a suspicious event has been detected, thereby preventing damage that may be caused by the moving object.

[0134] The notification performed by the notification control unit 160 may include, for example, displaying on the display unit 130 that a suspicious event has been detected. The notification performed by the notification control unit 160 may further include, for example, the details of the detected suspicious event. This allows the user to know that a suspicious event has been detected and to take appropriate action.

[0135] The display unit that the notification control unit 160 displays when notifying is not limited to the display unit 130 included in the information processing device 100, but may be included in, for example, a terminal device (not shown) connected via the communication network NT. The terminal device is, for example, a tablet terminal or a smartphone used by a user such as a resident.

[0136] The notification control unit 160 may also notify an external device (not shown) (for example, a device managed by a security company, a device managed by the police, a device managed by an automobile company, etc.) This allows the management company, police, automobile company, etc. to be notified that a suspicious event has occurred and can respond appropriately.

[0137] The notification method is not limited to the above example, and it is preferable that the notification be performed using at least one of these notification methods.

[0138] (Regarding the suspicious event storage unit 170) The suspicious event log stored in the suspicious event storage unit 170 is a history of detected suspicious events. The suspicious event log may include, for example, the time when the suspicious event was detected (occurrence time), the details of the suspicious event, etc.

[0139] (Example of Physical Configuration of Information Processing Device 100) As shown in FIG. 11, the information processing device 100 physically includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, an input interface 1060, and an output interface 1070.

[0140] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, network interface 1050, input interface 1060, and output interface 1070. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0141] The processor 1020 is implemented as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).

[0142] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0143] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read-only memory (ROM), or the like. The storage device 1040 stores program modules for realizing the functions of the device that includes the storage device 1040. The processor 1020 loads each of these program modules into the memory 1030 and executes them to realize the function corresponding to that program module.

[0144] The network interface 1050 is an interface for connecting a device equipped with it to a communication network.

[0145] The input interface 1060 is an interface for the user to input information, and is configured from, for example, a touch panel, a keyboard, a mouse, and the like.

[0146] The output interface 1070 is an interface for presenting information to the user, and is configured, for example, by a liquid crystal panel, an organic EL (Electro-Luminescence) panel, or the like.

[0147] In this way, the functions of the information processing device 100 can be realized by the physical components cooperating to execute a software program. Therefore, the present invention may be realized as a software program or as a non-transitory storage medium on which the program is recorded.

[0148] The in-vehicle device 30 may be physically configured in the same manner as the information processing device 100, for example.

[0149] The functions of the in-vehicle device 30 and the information processing device 100 as a whole are not limited to the above examples, and may be provided in one or more devices. For example, the in-vehicle device 30 may further include some or all of the functions of the information processing device 100.

[0150] (Operations and Effects) As described above, according to this embodiment, the information processing device 100 includes the generating unit 110 , the setting unit 140 , and the detecting unit 150 .

[0151] The generation unit 110 generates planar layout information showing in plan view the layout of a site area including a parking lot where automobiles are parked and a house, using original information obtained from at least one of GIS information, images taken by an on-board camera, and point cloud information generated using an on-board LiDAR sensor.

[0152] The setting unit 140 sets the monitoring area based on the designation for the planar layout information.

[0153] The detection unit 150 uses the analysis results of sensor information generated by the vehicle-mounted sensor to detect a predetermined suspicious event related to a moving object present in a monitoring area.

[0154] In this way, since the monitoring area is set using the layout of the site area, the on-board sensor can be used to monitor an appropriate monitoring area to ensure the safety of the home, thereby improving the safety of the home using the on-board sensor.

[0155] According to this embodiment, the detection unit 150 detects suspicious events using the analysis results and the behavior patterns of residents of the house.

[0156] This makes it possible to appropriately detect suspicious events that are likely to threaten the safety of a home by using resident behavior patterns, thereby improving safety in homes using vehicle-mounted sensors.

[0157] According to this embodiment, the setting unit 140 includes an automatic generation unit 141 and a monitoring area setting unit 142. The automatic generation unit 141 generates an automatically generated area for the site area. The monitoring area setting unit 142 sets a monitoring area based on the automatically generated area and specifications for the planar layout information.

[0158] This makes it possible to use the automatically generated area to assist in setting an appropriate monitoring area, thereby improving safety in homes using vehicle-mounted sensors.

[0159] According to this embodiment, the resident behavior pattern includes at least one of a route pattern relating to the resident's travel route and a time pattern relating to the resident's behavior time.

[0160] This makes it possible to appropriately detect suspicious events related to at least one of the travel route and the time of the activity that are likely to threaten the safety of the home by using the resident behavior patterns, thereby improving the safety of the home using in-vehicle sensors.

[0161] According to this embodiment, when the resident behavior pattern includes a route pattern, a suspicious event includes a moving object moving along a travel route different from the route pattern.

[0162] This allows a suspicious event to be detected when a moving object moves along a route that differs from the route pattern. Generally, when a moving object moves along a route that differs from the route pattern, this moving object is likely to pose a threat to the safety of a home. Therefore, by detecting such suspicious events, it is possible to improve the safety of a home using an in-vehicle sensor.

[0163] According to this embodiment, when the resident behavior pattern includes a time pattern, the detection unit 150 detects a suspicious event using the analysis results of sensor information obtained at the monitoring period set using the time pattern.

[0164] Generally, depending on the behavior of residents, which is often repeated over different time periods, there are time periods when monitoring of the monitoring area is less necessary and time periods when monitoring of the monitoring area is more necessary. By monitoring the monitoring area at the monitoring time set using a time pattern, the monitoring area can be monitored at the appropriate time. Therefore, it is possible to improve safety in homes using on-board sensors.

[0165] (Variation 1) In the first embodiment, an example has been described in which the in-vehicle device 30 performs the analysis process, but part or all of the analysis process may be performed by a device other than the in-vehicle device 30. In this case, the information processing system S2 may further include an analysis device 50 that performs part or all of the analysis process, as shown in Fig. 12, for example.

[0166] The in-vehicle device 30, the analysis device 50, and the information processing device 100 may be connected to one another via a communication network NT. The communication network NT may be configured as a wired or wireless network, or a combination of these. This allows the in-vehicle device 30, the analysis device 50, and the information processing device 100 to be connected to one another so that they can transmit and receive various types of information to and from one another.

[0167] It should be noted that the multiple on-board sensors 20, on-board devices 30, analysis devices 50, and information processing devices 100 only need to be connected so that they can send and receive information to each other, and the configuration for connecting them is not limited to the example described above.

[0168] This modification also provides the same effects as the first embodiment.

[0169] [Embodiment 2] In embodiment 1, an example was described in which a resident or the like sets a resident behavior pattern. A resident behavior pattern may be created automatically. In embodiment 2, an example is described in which a resident behavior pattern is automatically created using the behavior history of the resident. In this embodiment, for simplicity, explanations that overlap with embodiment 1 will be omitted as appropriate.

[0170] (Configuration Example of Information Processing Apparatus 200) As shown in FIG. 13, for example, the information processing apparatus 200 includes a pattern creating unit 210 in addition to the components included in the information processing apparatus 100.

[0171] The pattern creation unit 210 creates a resident behavior pattern using the behavior history of the resident.

[0172] (Example of Operation of Information Processing Device 200) The pattern creating unit 210 executes a pattern creating process as shown in FIG. 14, for example.

[0173] The pattern creation unit 210 creates a resident behavior pattern using the behavior history of the resident (step S210).

[0174] (Regarding the Pattern Creation Unit 210) The pattern creation unit 210, for example, acquires the analysis results in real time and stores the behavior history of the resident included in the analysis results.

[0175] At this time, the pattern creation unit 210 may use predetermined family information to identify and store the behavior history of the resident from the analysis results.

[0176] The pattern creation unit 210 may also identify a person who is detected with a predetermined frequency or more from the analysis results for a predetermined period or more as a resident, and store the behavior history of this identified resident. In this case, the pattern creation unit 210 may automatically create and set the above-mentioned resident information.

[0177] The pattern creation unit 210 executes a pattern creation process at an appropriate timing (for example, a predetermined cycle such as daily, weekly, or monthly) using the stored behavior history of the resident. This allows a resident behavior pattern similar to that of embodiment 1 to be created. Note that the information processing device 200 may execute the information processing shown in FIG. 8, and this pattern creation process may be included as part of the information processing executed by the information processing device 200.

[0178] The pattern creation unit 210 may create a resident behavior pattern for each resident, for example, by using the stored behavior history and resident information of the resident. The pattern creation unit 210 may store the resident behavior pattern, or the detection unit 150 or the like may store the resident behavior pattern.

[0179] The resident behavior pattern created by the pattern creation unit 210 may be used in the same manner as the resident behavior pattern described in the first embodiment.

[0180] (Operations and Effects) As described above, according to this embodiment, the information processing device 200 further includes the pattern creation unit 210 that creates a resident behavior pattern using the behavior history of the resident.

[0181] This allows resident behavior patterns to be created automatically. Therefore, it is possible to appropriately detect suspicious events that are likely to threaten the safety of a home using resident behavior patterns while eliminating the need to set up resident behavior patterns. Therefore, it is possible to improve safety in a home using in-vehicle sensors while eliminating the need to set up resident behavior patterns.

[0182] [Embodiment 3] When a suspicious event is detected, if all suspicious events are displayed or notified on the display unit 130 or a terminal device, there is a possibility that displays and notifications will be frequently made, including suspicious events that are relatively less dangerous. In such a case, displays and notifications may be neglected, which may result in the display and notification not being useful in improving safety in the home.

[0183] In the third embodiment, an example will be described in which suspicious events are ranked and a display or notification is performed according to the rank.

[0184] In this embodiment, for the sake of simplicity, descriptions that overlap with those in the first embodiment will be omitted as appropriate.

[0185] (Configuration example of information processing device 300) For example, as shown in Fig. 15 , the information processing device 300 includes a notification control unit 360 instead of the notification control unit 160. The information processing device 300 further includes an event classification unit 310. Except for these components, the information processing device 300 may be configured similarly to the information processing device 100 shown in Fig. 7 , for example.

[0186] The event classification unit 310 classifies the detected suspicious event into a plurality of ranks based on the analysis result of the mobile object in which the suspicious event was detected.

[0187] The notification control unit 360 notifies the detected suspicious event based on the rank.

[0188] (Example of Operation of Information Processing Device 300) The information processing device 300 executes information processing such as that shown in FIG. 16, for example.

[0189] Steps S110 to S150 similar to those in the first embodiment are executed.

[0190] The event classification unit 310 classifies the detected suspicious event into a plurality of ranks based on the analysis result of the mobile object in which the suspicious event was detected (step S310).

[0191] The notification control unit 360 notifies the user of the detected suspicious event based on the rank (step S360).

[0192] Step S170 is executed in the same manner as in the first embodiment.

[0193] (Regarding the Event Classification Unit 310) When the detection unit 150 detects a suspicious event, the event classification unit 310 classifies the detected suspicious event into a plurality of ranks based on an analysis result regarding the mobile object in which the suspicious event was detected. The analysis result regarding the mobile object in which the suspicious event was detected is, for example, the analysis result used to detect the suspicious event.

[0194] The plurality of ranks may be ranks according to predetermined risk levels, and the rank into which a suspicious event is classified may be determined in advance for each suspicious event.

[0195] For example, in the above example of suspicious events, peering and crouching under a vehicle may be set as high-risk suspicious events. For example, a moving object traveling from a gate to a front door via a route that is not the shortest route from the gate to the front door may be set as a high-risk suspicious event.

[0196] For example, loitering, peering, and crouching anywhere other than under the vehicle, other than moving at a low speed for a predetermined period of time, may be set as suspicious events with a medium level of risk.

[0197] For example, loitering, which involves moving slowly for a predetermined period of time or longer, may be set as a suspicious event with a low risk level.

[0198] For example, if a low-risk or medium-risk suspicious event is detected at night, the suspicious event may be configured to be classified as a medium-risk or high-risk suspicious event, respectively. For example, if a medium-risk suspicious event is detected while the resident is not present at the residence, the suspicious event may be configured to be classified as a high-risk suspicious event.

[0199] For example, if the above-mentioned moderate suspicious event is detected while only children or elderly people are present in the dwelling, the suspicious event may be set to be classified as a high-risk suspicious event. For example, if loitering other than moving at a low speed for a predetermined period of time is detected as a suspicious event during a predetermined time period when there is a lot of human movement, the suspicious event may be set to be a low-risk suspicious event.

[0200] In this way, the rank of a suspicious event may be determined using at least one of the following: the time of day when the suspicious event was detected, the presence or absence of residents in the home at the time the suspicious event was detected, the attributes of the residents in the home at that time, etc.

[0201] For example, if the height of a moving object moving at a low speed for a predetermined period of time is equal to or greater than a threshold, the suspicious event may be classified as a suspicious event of medium risk. In this manner, the rank of the suspicious event may be determined using the attributes of the moving object in which the suspicious event was detected.

[0202] (Regarding the Notification Control Unit 360) The notification control unit 360 notifies of a detected suspicious event based on the rank classified by the event classification unit 310.

[0203] For example, the notification control unit 360 may issue a predetermined notification depending on the rank of the detected suspicious event. For example, for a suspicious event with a high risk rank, the notification control unit 360 may issue a warning to the mobile object, display on the display unit 130, and notify an external device. For example, for a suspicious event with a medium risk rank, the notification control unit 360 may issue a warning to the mobile object and display on the display unit 130. For example, for a suspicious event with a low risk rank, the notification control unit 360 may not issue a notification. Even in this case, the suspicious event may be stored in the suspicious event log.

[0204] (Actions and Effects) As described above, according to this embodiment, the information processing device 300 further includes an event classification unit 310 that classifies a detected suspicious event into multiple ranks based on the analysis results regarding the mobile body in which the suspicious event was detected.

[0205] This allows appropriate responses to be taken depending on the rank of the detected suspicious event, thereby improving safety in the home.

[0206] According to this embodiment, the information processing device 300 further includes a notification control unit 360 that notifies of a detected suspicious event based on the rank.

[0207] This allows a notification to be made according to the rank of the detected suspicious event and to be sent to a notification destination according to the rank of the suspicious event, thereby improving safety in the home.

[0208] [Fourth Embodiment] In the first embodiment, an example was described in which a predetermined suspicious event related to a moving object present in a monitoring area is detected using the analysis results of sensor information generated by an on-board sensor. To detect such a suspicious event, the analysis results of sensor information generated by a sensor other than an on-board sensor may also be used. In the fourth embodiment, an example is described in which the analysis results of sensor information generated by a residential sensor, which is a sensor installed on a premises, is also used as a sensor other than an on-board sensor.

[0209] In this embodiment, for the sake of simplicity, descriptions that overlap with those in the first embodiment will be omitted as appropriate.

[0210] 17 , the information processing system S3 includes a plurality of on-board sensors 20 and an on-board device 30 mounted on an automobile 10 similar to those in the first embodiment, and an information processing device 100. The information processing system S3 further includes an analysis device 51 and a plurality of home sensors 21. Note that the number of home sensors 21 may be one.

[0211] Each of the home sensors 21 is a sensor installed on the premises to detect objects inside or around the home. Each of the multiple home sensors 21 generates sensor information indicating objects inside or around the home. By analyzing this sensor information, it is possible to detect objects inside or around the home.

[0212] The plurality of home sensors 21 may be composed of at least one of, for example, a camera installed in the home, a LiDAR (Light Detection and Ranging) sensor installed in the home, an object presence detection sensor installed in the home, and the like.

[0213] The analysis device 51 acquires sensor information generated by each of the plurality of home sensors 21, analyzes the acquired sensor information, and generates an analysis result. The analysis function of the analysis device 51 may be similar to that of the analysis unit 32.

[0214] The analysis device 51 may have the same functions as the on-vehicle device 30 and analyze the sensor information generated by each of the multiple on-vehicle sensors 20. The analysis device 51 may generate an analysis result that combines the individual analysis results of the sensor information generated by each of the multiple on-vehicle sensors 20 and the individual analysis results of the sensor information generated by each of the multiple home sensors 21. The functions of the analysis device 51 may be provided in the on-vehicle device 30.

[0215] In this embodiment, the detection unit 150 may detect a predetermined suspicious event related to a moving object present in the monitoring area by using the analysis results of the sensor information generated by the on-board sensor 20 and the house sensor 21. Furthermore, in step S150, the detection unit 150 may detect a predetermined suspicious event related to a moving object present in the monitoring area by using the analysis results of the sensor information generated by the on-board sensor 20 and the house sensor 21.

[0216] (Actions and Effects) As described above, according to this embodiment, the detection unit 150 detects predetermined suspicious events related to moving objects present in the monitoring area by using the analysis results of the sensor information generated by the vehicle-mounted sensor 20 and the sensor information generated by the residential sensor 21 installed on the premises.

[0217] This allows suspicious events to be detected using the house sensor 21 used in a typical home security system and the vehicle-mounted sensor 20. Therefore, suspicious events can be detected over a wider area or with higher accuracy than when only the vehicle-mounted sensor 20 is used. Therefore, it is possible to further improve safety in homes by using the vehicle-mounted sensor.

[0218] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0219] In addition, although the flowcharts used in the above description show a sequence of steps (processes), the order of steps executed in each embodiment is not limited to the sequence shown in the flowcharts. In each embodiment, the order of steps shown in the diagrams can be changed as long as it does not cause any problems in terms of the content.

[0220] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0221] 1. An information processing device comprising: a planar layout generation means for generating planar layout information showing in plan view the layout of a site area including a parking lot where automobiles are parked and a house, using original information obtained from at least one of GIS information, images captured by an on-board camera, and point cloud information generated using an on-board LiDAR sensor; a setting means for setting a monitoring area based on specifications for the planar layout information; and a detection means for detecting a predetermined suspicious event related to a moving object present in the monitoring area using an analysis result of sensor information generated by the on-board sensor. 2. The information processing device described in 1., wherein the detection means detects the suspicious event using the analysis result and behavior patterns of residents of the house. 3. The information processing device described in 1. or 2., further comprising an event classification means for classifying the detected suspicious event into a plurality of ranks based on the analysis result related to the moving object in which the suspicious event was detected. 4. The information processing device according to any one of 1 to 3, wherein the setting means includes: an automatic generation means for generating an automatically generated area for the site area; and a monitoring area setting means for setting the monitoring area based on the automatically generated area and the specification for the planar layout information. 5. The information processing device according to any one of 2 to 4, wherein the resident behavior pattern includes at least one of a route pattern related to the resident's movement route and a time pattern related to the resident's movement time. 6. The information processing device according to 5., wherein, when the resident behavior pattern includes the route pattern, the suspicious event includes the mobile object moving along a movement route different from the route pattern. 7. The information processing device according to 5 or 6, wherein, when the resident behavior pattern includes the time pattern, the detection means detects the suspicious event using the analysis result of the sensor information obtained at a monitoring period set using the time pattern. 8. The information processing device according to any one of 2 and 5 to 7, further comprising pattern creation means for creating the resident behavior pattern using the resident's behavior history. 9. 3. The information processing device according to Item 3, further comprising a notification control unit that notifies the user of the detected suspicious event based on the rank.10. The information processing device described in any one of 1. to 9., wherein the on-board sensor includes at least one of the on-board camera, the on-board LiDAR sensor, and an object presence detection sensor for detecting the presence or absence of an object in the vicinity of the vehicle. 11. The information processing device described in any one of 1. to 10., wherein the detection unit means detects a predetermined suspicious event related to a moving object present in a monitoring area using an analysis result of sensor information generated by the on-board sensor and sensor information generated by a residential sensor installed on a premises. 12. An information processing system comprising the on-board sensor and on-board device mounted on the vehicle, and the information processing device described in any one of 1. to 11., wherein the on-board device, upon acquiring the sensor information from the on-board sensor, performs analysis using the sensor information and transmits the analysis result to the information processing device. 13. 14. An information processing method in which one or more computers generate planar layout information showing, in plan view, the layout of a site area including a parking lot where automobiles are parked and a house, using original information obtained from at least one of GIS information, images captured by an on-board camera, and point cloud information generated using an on-board LiDAR sensor, set a monitoring area based on specifications for the planar layout information, and detect predetermined suspicious events related to moving objects present in the monitoring area using analysis results of sensor information generated by the on-board sensor. 14. The information processing method described in 13., in which detecting the suspicious event includes detecting the suspicious event using the analysis results and behavior patterns of residents of the house. 15. The information processing method described in 13. or 14., further including classifying the detected suspicious event into a plurality of ranks based on the analysis results related to the moving object in which the suspicious event was detected. 16. 16. The information processing method according to any one of 13 to 15, wherein setting the monitoring area includes: generating an automatically generated area for the site area; and setting the monitoring area based on the automatically generated area and the specification for the planar layout information.17. The information processing method according to any one of 14 to 16, wherein the resident behavior pattern includes at least one of a route pattern relating to the resident's movement route and a time pattern relating to the resident's behavior time. 18. The information processing method according to 17, wherein, when the resident behavior pattern includes the route pattern, the suspicious event includes the mobile object moving along a movement route different from the route pattern. 19. The information processing method according to 17 or 18, wherein, when the resident behavior pattern includes the time pattern, the suspicious event is detected using the analysis results of the sensor information obtained at a monitoring period set using the time pattern. 20. The information processing method according to any one of 14 and 17 to 19, further including creating the resident behavior pattern using the resident's behavior history. 21. The information processing method according to 15, further including notifying the detected suspicious event based on the rank. 22. The information processing method according to any one of 13. to 21., wherein the on-board sensor includes at least one of the on-board camera, the on-board LiDAR sensor, and an object presence / absence detection sensor for detecting the presence or absence of an object in the vicinity of the vehicle. 23. The information processing method according to any one of 13. to 22., wherein the detecting a suspicious event includes detecting a predetermined suspicious event related to a moving object present in a monitoring area using analysis results of sensor information generated by the on-board sensor and sensor information generated by a residential sensor installed on the premises. 24. A program causing one or more computers to execute the following steps: generate planar layout information showing, in a plan view, a layout of a site area including a parking lot where automobiles are parked and a residential area using original information obtained from at least one of GIS information, images captured by an on-board camera, and point cloud information generated using an on-board LiDAR sensor; set a monitoring area based on specifications for the planar layout information; and detect a predetermined suspicious event related to a moving object present in the monitoring area using analysis results of sensor information generated by the on-board sensor.25. The program according to 24., wherein detecting the suspicious event detects the suspicious event using the analysis result and a resident behavior pattern of the residence. 26. The program according to 24. or 25., further causing the program to classify the detected suspicious event into a plurality of ranks based on the analysis result regarding the mobile object in which the suspicious event was detected. 27. The program according to any one of 24 to 26, wherein setting the monitoring area includes generating an automatically generated area regarding the site area, and setting the monitoring area based on the automatically generated area and the specification for the planar layout information. 28. The program according to any one of 25 to 27, wherein the resident behavior pattern includes at least one of a route pattern regarding the resident's movement route and a time pattern regarding the resident's activity time. 29. The program according to 28., wherein, when the resident behavior pattern includes the route pattern, the suspicious event includes the mobile object moving along a movement route different from the route pattern. 30. The program according to 28 or 29, wherein detecting the suspicious event includes, when the resident behavior pattern includes the time pattern, detecting the suspicious event using the analysis result of the sensor information obtained at a monitoring period set using the time pattern. 31. The program according to any one of 25 and 28 to 30, which is further configured to create the resident behavior pattern using the behavior history of the resident. 32. The program according to 26, which is further configured to notify the detected suspicious event based on the rank. 33. The program according to any one of 24 to 32, wherein the on-board sensor includes at least one of the on-board camera, the on-board LiDAR sensor, and an object presence detection sensor for detecting the presence or absence of an object in the vicinity of the vehicle.34. The program according to any one of 24. to 33., wherein detecting the suspicious event includes detecting a predetermined suspicious event related to a moving object present in a monitoring area using an analysis result of sensor information generated by the vehicle-mounted sensor and sensor information generated by a residential sensor installed on the premises. 35. A recording medium having recorded thereon the program according to any one of 24. to 34.

[0222] This application claims priority based on Japanese Patent Application No. 2024-003143, filed January 12, 2024, the disclosure of which is incorporated herein by reference in its entirety.

[0223] REFERENCE SIGNS LIST 10 Automobile 20 In-vehicle sensor 21 Residential sensor 30 In-vehicle device 31 Sensor information acquisition unit 32 Analysis unit 50, 51 Analysis device 100, 200, 300 Information processing device 110 Generation unit 120 Display control unit 130 Display unit 140 Setting unit 141 Automatic generation unit 142 Monitoring area setting unit 150 Detection unit 160, 360 Notification control unit 170 Suspicious event storage unit 210 Pattern creation unit 310 Event classification unit

Claims

1. A plane layout generation means for generating plane layout information showing a layout of a site area including a parking lot where an automobile parks and a house in a plan view, using original information obtained from at least one of GIS information, an image captured by an in-vehicle camera, and point cloud information generated using an in-vehicle LiDAR sensor; a setting means for setting a monitoring area based on a designation for the plane layout information; and a detection means for detecting a predetermined suspicious event regarding a moving object existing in the monitoring area using an analysis result of sensor information generated by an in-vehicle sensor. An information processing apparatus.

2. The information processing apparatus according to claim 1, wherein the detection means detects the suspicious event using the analysis result and the movement pattern of the residents of the house.

3. The information processing apparatus according to claim 1 or 2, further comprising an event classification means for classifying the detected suspicious event into a plurality of ranks based on the analysis result regarding the moving object in which the suspicious event is detected.

4. The information processing apparatus according to claim 2, wherein the movement pattern of the residents includes at least one of a route pattern regarding the movement route of the residents and a time pattern regarding the movement time of the residents.

5. The information processing apparatus according to claim 4, wherein when the movement pattern of the residents includes the route pattern, the suspicious event includes the moving object moving along a movement route different from the route pattern.

6. The information processing apparatus according to claim 4, wherein when the movement pattern of the residents includes the time pattern, the detection means detects the suspicious event using the analysis result of the sensor information obtained during a monitoring period set using the time pattern.

7. The information processing apparatus according to claim 2, further comprising a pattern creation means for creating the movement pattern of the residents using the movement history of the residents.

8. The information processing apparatus according to claim 3, further comprising a notification control means for notifying the detected suspicious event based on the rank.

9. An information processing method in which one or more computers generate plane layout information showing a layout of a site area including a parking lot where a vehicle parks and a house in a plan view using original information obtained from at least one of GIS information, an image captured by an in-vehicle camera, and point cloud information generated using an in-vehicle LiDAR sensor, set a monitoring area based on a specification for the plane layout information, and detect a predetermined suspicious event regarding a moving object present in the monitoring area using an analysis result of sensor information generated by an in-vehicle sensor.

10. A recording medium having recorded thereon a program for causing one or more computers to generate plane layout information showing a layout of a site area including a parking lot where a vehicle parks and a house in a plan view using original information obtained from at least one of GIS information, an image captured by an in-vehicle camera, and point cloud information generated using an in-vehicle LiDAR sensor, set a monitoring area based on a specification for the plane layout information, and detect a predetermined suspicious event regarding a moving object present in the monitoring area using an analysis result of sensor information generated by an in-vehicle sensor.

Images