Information processing device, information processing method, program, and information processing system

Abstract

The system automatically detects changes in the environment surrounding the path the person being monitored is taking, and if those changes suggest potential danger, it determines the degree of that danger and notifies the caregiver with specific information. This helps ensure the safety of the person being monitored and contributes to reducing the caregiver's burden and improving their sense of security. [Solution] The information processing device receives image data and location information from a camera-equipped terminal for the person being monitored, and identifies past comparison image data based on the received location information. It compares the current image data with past image data to detect environmental changes and determines the potential risk level based on those changes. Based on the determined risk level, it transmits information regarding environmental changes to the monitor's terminal. This enables early detection of potential risks caused by environmental changes along the travel route and provides appropriate notification.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, a program, and an information processing system.

Background Art

[0002] In recent years, various technologies have been proposed for monitoring the safety of children, the elderly, and the like. For example, the following patent documents disclose a position monitoring system that uses the Global Positioning System (GPS) to identify the current position of a person under surveillance and allows a caregiver or the like to check the position information.

[0003] Furthermore, wearable devices equipped with cameras and smartwatches for children have also emerged. Some of these products transmit snapshot images of the surroundings of the person under surveillance or provide a limited video call function (see, for example, Non-Patent Document 1). As a result, in addition to the position information, caregivers can fragmentarily obtain visual information about the surroundings of the person under surveillance.

Prior Art Documents

Patent Documents

[0004] Japanese Unexamined Patent Application Publication No. 2020-161906

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, the conventional technologies described above are still insufficient in that they continuously capture changes in the environment along the routes that the person being monitored, especially a child, travels on a daily basis, and proactively and specifically warn the monitor when those changes suggest potential danger.

[0007] The present invention aims to provide an information processing device, information processing method, program, and information processing system that can support the safety of a person being monitored and contribute to reducing the burden on the monitor and improving their sense of security by automatically detecting changes in the environment around the path a person being monitored travels, based on a comparison with past conditions, determining the degree of danger when such changes suggest potential danger, and notifying the monitor with specific information. [Means for solving the problem]

[0008] To solve the above problems, an information processing device according to one aspect of the present invention is: A receiving unit that receives image data and location information transmitted from a monitoring terminal equipped with a camera, A past data identification unit identifies past image data for comparison, which is past image data of the movement route of the person being monitored, based on the received location information. A change detection unit compares the received image data with the identified past image data for comparison and detects changes in the environment surrounding the travel path based on the difference between the two. A determination unit that determines the degree of potential danger to the person being monitored based on the detected environmental changes, A transmitting unit transmits information regarding the environmental changes to a monitoring terminal based on the determined degree of danger. It holds. [Effects of the Invention]

[0009] According to one aspect of the present invention, it is possible to detect potential dangers caused by changes in the environment around the path a person being monitored travels at an early stage and to appropriately notify the person of such dangers. As a result, the person being monitored can receive support to more effectively ensure the safety of the person being monitored, reducing the burden of daily safety checks and increasing their sense of security. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram showing the overall configuration of a risk level notification system according to one embodiment of the present invention. [Figure 2] This is a block diagram showing an example of the hardware configuration of the monitoring terminal according to this embodiment. [Figure 3] This is a block diagram showing an example of the hardware configuration of a server (information processing device) according to this embodiment. [Figure 4] This is a functional block diagram showing the main functional configuration of the server (information processing device) according to this embodiment. [Figure 5] This flowchart shows the overall flow of the risk level notification process in this embodiment. [Figure 6] This flowchart shows an example of a process for identifying past data. [Figure 7] This flowchart shows an example of change detection processing. [Figure 8] This is a flowchart showing an example of a risk assessment process. [Figure 9] This is a screenshot showing an example of notification information displayed on a monitoring device. [Figure 10] This is a screenshot showing an example of a risk level map displayed on a monitoring device. [Figure 11] This is a screenshot showing an example of the privacy settings screen. [Figure 12] This is a table diagram showing an example of the information stored in the database. [Figure 13] This is a screen diagram showing a specific example of how the risk map according to this embodiment is displayed. [Embodiment for Carrying Out the Invention]

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each figure, the same or corresponding components are denoted by the same reference numerals, and redundant descriptions will be omitted as appropriate. Also, in this specification and the drawings, components with the same reference numerals should be understood to have the same configuration or to exhibit the same function. Furthermore, the configurations described in each embodiment can be appropriately combined within a range where there is no technical contradiction.

[0012] [System Configuration] FIG. 1 is a schematic diagram showing the overall configuration of a risk notification system 1 according to an embodiment of the present invention. The risk notification system 1 mainly includes a ward-end terminal 10 for the ward M that the ward M carries or wears, a server 20 that functions as an information processing device, and a watcher-end terminal 30 that is used by a watcher U such as a guardian of the ward M. These ward-end terminals 10, server 20, and watcher-end terminals 30 are communicably connected to each other via a network 40 that combines one or more of the Internet, a mobile phone network, a wireless LAN (Local Area Network), Bluetooth (registered trademark), and the like. The ward M is mainly assumed to be a child, but may also be an elderly person or other person who requires assistance in ensuring safety. The watcher U is usually a parent or guardian of the ward M, but may also be a school-related person or a caregiver, etc. The server 20 may be composed of a single physical server, or may be realized in a cloud computing environment in which functions are distributed among a plurality of servers.

[0013] [Ward-End Terminal 10] FIG. 2 is a block diagram showing an example of the hardware configuration of the wardee terminal 10. The wardee terminal 10 is configured as, for example, a smartphone, a smartwatch, a pendant-type dedicated device, or a small wearable device that can be attached to a mobile phone or a bag. More specifically, the wardee terminal 10 may be configured as a compact flat solid that fits in a child's palm. In this case, the contour shape of the main surface (e.g., the front surface) of the terminal can adopt various shapes such as a substantially square, rectangular, circular, or elliptical shape, or the entire terminal may be in a stick-like elongated shape like a pen. An operation means for constituting a user interface can be arranged at the center of such a main surface or at a position suitable for operation. This operation means may be, for example, a composite operation unit (e.g., circular or any shape according to the shape of the main surface) that integrally has a display function, a touch input function, and a button function capable of physical pressing operation. Alternatively, components that provide these functions, namely a display unit (e.g., a liquid crystal display or an organic EL display) having a display function, a touch panel having a touch input function, and at least one physical operation button (e.g., a power button, an SOS button, other function buttons) may be provided at appropriate positions on the housing as independent parts or in a combination of some of them. The housing may further have a microphone hole for voice input and a charging connector (e.g., a USB Type-C (registered trademark) terminal, etc.) provided at appropriate positions, and a rechargeable battery may be built in. The wardee terminal 10 includes a processor 101, a memory 102, a storage unit 103, a communication module 104, a camera 105, a GPS receiver 106, an IMU (Inertial Measurement Unit) 107, a voice input / output unit 108, and a battery 109, etc.

[0014] The processor 101 is a CPU (Central Processing Unit) or MPU (Micro Processing Unit), and comprehensively controls the operation of the entire monitored terminal 10 by reading programs and data stored in the storage unit 103 into the memory 102 and executing them. The memory 102 includes RAM (Random Access Memory) and ROM (Read Only Memory), and functions as the work area and program storage area for the processor 101. The storage unit 103 is a non-volatile storage such as flash memory, and stores the OS (Operating System), various application programs, captured image data, location information, sensor data, etc.

[0015] The communication module 104 is an interface for wireless communication with the server 20 and, in some cases, the monitor terminal 30 via the network 40. For example, it supports communication standards such as LTE (Long Term Evolution), 5G (5th Generation mobile communication system), Wi-Fi (Wireless Fidelity), and Bluetooth (registered trademark). The camera 105 is a digital camera for photographing the direction of movement and the surrounding environment of the person being monitored M, and is capable of capturing video or still images. It is desirable that the camera 105 be fixed to the shoulder strap of a school bag, clothing, a hat, etc., or designed to be held in such an orientation, so that it can stably photograph the front from a position close to the eye level of the person being monitored M. In the case of the flat three-dimensional housing as described above, the camera 105 may be placed one or more on one main surface of the housing (for example, the back where no operating means or display is located), or on both opposing main surfaces (for example, both the front and back where the composite operating unit is located). The camera 105 is preferably positioned away from the control panel, particularly above the control panel or near the corners of the housing, to prevent the lens from being covered by the user's fingers while operating the control panel. This allows for stable shooting even during operation. This enables the acquisition of images close to the perspective of the person being monitored M, contributing to improved accuracy of the AI-based risk assessment described later. The GPS receiver 106 receives signals from GPS satellites to determine the current location (latitude, longitude, altitude, etc.) of the monitored terminal 10. The IMU 107 includes an accelerometer, gyroscope, and geomagnetic sensor to detect the movement, orientation, and posture of the monitored terminal 10. The audio input / output unit 108 includes a microphone and speaker and is used for recording control via voice commands and for limited communication functions (such as when sending an SOS). The battery 109 supplies power to various parts of the monitored terminal 10.

[0016] [Server 20 (Information Processing Device)] Figure 3 is a block diagram showing an example of the hardware configuration of server 20. Server 20 is configured as a general computer system, comprising a processor 201, memory 202, storage device 203, communication interface 204, and input / output interface 205, which are connected via a bus 206. The processor 201 controls the operation of the entire server 20 and realizes the processing of each functional unit described later by loading programs stored in storage device 203 into memory 202 and executing them. Memory 202 includes RAM and ROM. Storage device 203 is a large-capacity storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), and stores the OS, a database management system (DBMS), programs for executing various processes according to the present invention, and databases described later (for example, user information DB, image / location information DB 301 (see Figure 12), past situation DB, risk map DB, etc.). The communication interface 204 communicates with the monitored terminal 10 and the monitor terminal 30 via the network 40. Input / output interface 205 allows connection of input / output devices such as displays, keyboards, and mice, and is used for system management and maintenance.

[0017] [Monitoring device 30] The monitoring terminal 30 is an information terminal that the monitor U normally uses, such as a smartphone, tablet, or personal computer. The monitoring terminal 30 is equipped with a processor, memory, storage unit, communication module, display, and input unit such as a touch panel or keyboard, and by running a dedicated application (hereinafter referred to as the "monitoring app"), it provides functions such as receiving and displaying danger notification information transmitted from the server 20, displaying the location information of the person being monitored M on a map, and making various settings.

[0018] [Functional Configuration and Operation] Next, the functional configuration and operation of this embodiment, centered on the server 20, will be described with reference to the functional block diagram in Figure 4 and the flowcharts in Figures 5 to 8. As shown in Figure 4, the server 20 mainly comprises a receiving unit 21, a past data identification unit 22, a change detection unit 23, a determination unit 24, and a transmission unit 25. In addition to these, this embodiment will be described as further comprising an action detection unit 26, a map processing unit 27, an image correction unit 28, an anonymization processing unit 29, and a consent management unit (not shown). However, these additional functional units are not essential and can be appropriately selected or omitted depending on the implementation of the invention.

[0019] (1) Receiving unit 21 The receiving unit 21 receives data transmitted from the monitored person terminal 10 via the network 40. Specifically, it mainly receives image data captured by the camera 105 of the monitored person terminal 10 (for example, still image data in formats such as JPEG and PNG, or video data encoded in formats such as MPEG and H.264), location information identified by the GPS receiver 106 (including latitude, longitude, altitude, and time information), and sensor data detected by the IMU 107 (acceleration, angular velocity, direction, etc.) (Figure 5: S201). The receiving unit 21 adds a timestamp to the received data, associates it with a unique identifier of the monitored person M (such as a user ID), and stores it chronologically in a database on the server 20 (for example, the image / location information DB 301 shown in Figure 12).

[0020] (2) Past data identification unit 22 The past data identification unit 22 identifies comparison past image data, which is past image data along the movement path of the person being monitored M, based on the current location information received by the receiving unit 21 (Figure 6: S601) (see Figure 5: S202, Figure 6). This comparison past image data serves as a reference for detecting changes in the environment by comparing it with the current image data.

[0021] First, the past data identification unit 22 identifies past image data that spatially corresponds to the current location information. Specifically, it searches the database 301 for image data previously taken at the same location as the current location information or at a nearby location (for example, a location within a preset acceptable range, such as within a radius of 5 meters centered on the current location) (S602). It is desirable that this search considers not only the location information but also the orientation of the camera (for example, the orientation information obtained from the IMU 107).

[0022] Next, the past data identification unit 22 selects candidate past image data suitable for comparison, taking into account the periodicity of the monitored person M's behavior or the stability of the recent environment (S603). Considering the periodicity of behavior means, for example, that traffic volume, pedestrian traffic, presence or absence of parked vehicles on the street, and the opening status of shops can differ significantly between weekday morning and evening school / kindergarten hours and daytime or holiday situations. Therefore, it prioritizes selecting past image data from the same time period and day of the week that corresponds to the current time and day of the week. This makes it possible to compare with patterns that are repeated on a daily basis, and to capture "unusual" changes with higher accuracy.

[0023] Considering the stability of the immediate environment means, for example, that if road construction is being carried out in the same location for several days, the conditions during the construction can be considered a temporary "normal" state. The past data identification unit 22 refers to data from the same time and location over the past few days (for example, the past 3 to 7 days), and if conditions such as weather are similar, it considers them as candidates for comparison (part of S603). This allows for the detection of changes such as immediately after the start or end of short-term construction, while preventing the false detection of stable conditions during the construction period as "changes" each time.

[0024] These considerations allow the historical data identification unit 22 to better handle changes that might be missed or falsely detected by a simple comparison with recent historical data, resulting in improved quality of notifications to parents and reduced burden from unnecessary alerts.

[0025] Furthermore, the past data identification unit 22, if necessary (if "Yes" is determined in S604), generates data representing representative past conditions at the location from multiple past time points of image data selected as described above (S605), and this can be used as comparative past image data. When referring to a single past image, temporary objects that happen to be captured in that image (e.g., a passing bicycle, debris blown by the wind) or temporary changes in lighting conditions (e.g., sunlight through a break in the clouds) can become noise, potentially reducing the accuracy of change detection. By generating data representing representative past conditions, the influence of such noise can be reduced, and a more stable comparison standard can be constructed.

[0026] One method for generating data that represents this typical past situation (S605) is to extract image features (for example, local feature descriptors such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), or deep features using CNN (Convolutional Neural Network), etc.) from image data at multiple past points in time, and then statistically process these features (for example, calculating the mean vector of each feature descriptor, performing feature clustering and extracting representative cluster centers, etc.). This generates a representative feature description that captures the structural features of the scene, and by comparing this with the feature description extracted from the current image, more meaningful changes can be detected.

[0027] Alternatively, one could select one or more representative images from image data at multiple past points in time, or generate a representative image by synthesizing partially meaningful regions from multiple images (S605). In selecting a representative image, for example, the similarity between images (e.g., histogram similarity, number of feature point matches, etc.) could be calculated, and the image that is most similar to the most images or the image that is evaluated as having less noise could be selected as representative. In image synthesis, for example, techniques such as background modeling can be applied to generate an image that closely resembles the background with temporary moving objects and noise removed by aligning multiple images and taking the median or average of the pixel values.

[0028] The historical data identification unit 22 can construct more accurate comparison criteria (for example, "a typical scene of intersection A on a rainy weekday morning") by applying the representative historical situation data generation process (S605) to a set of historical image data that matches specific days of the week, time of day, weather conditions, etc. This improves the ability to extract truly noteworthy changes while effectively absorbing fluctuations due to environmental factors, and a synergistic effect that significantly suppresses false detections can be expected.

[0029] Furthermore, if the monitored person M is visiting a new location for the first time, or immediately after starting to use the system, and there is no past image data of the monitored person in the database, or if there is not enough data of sufficient quantity or quality accumulated (resulting in a "No" in S606), the past data identification unit 22 can utilize image data collected from other monitored persons (with prior consent and in a manner that protects personal information) and stored on the cloud server 20C. Specifically, it searches for anonymized image data previously recorded by other users that matches the current location information and direction of travel (including contextual information such as time and weather if necessary), and identifies it as comparison past image data (S607). This allows the user to benefit from a certain level of environmental change detection functionality from the initial stages of using the system, or even in unfamiliar locations. This function is effective in improving the convenience of the system and providing a valuable service to more users. Finally, comparison past image data is identified through these processes (S608).

[0030] (3) Processing of the change detection unit 23 The change detection unit 23 receives the current image data input from the receiving unit 21 and the past image data for comparison identified by the past data identification unit 22 (Figure 7: S701), compares them, and detects changes in the environment surrounding the movement path of the person being monitored M based on the difference between them (see Figure 5: S203, Figure 7).

[0031] To accurately detect changes, it is first necessary to correct for differences in viewpoint and shooting conditions between the current image data and past image data used for comparison. The change detection unit 23 performs high-precision alignment processing (S702). In this process, the approximate position and orientation are estimated using GPS information and IMU sensor (accelerometer, gyroscope) information obtained from the monitored person's terminal 10. Furthermore, computer vision techniques such as feature point matching between images (for example, using feature descriptors such as SIFT, SURF, ORB) and optical flow analysis are applied to estimate geometric deformations (translation, rotation, scaling, distortion, etc.) between the two images, and one image is aligned with the other. This minimizes apparent differences caused by camera movement and changes in viewpoint, and improves the accuracy of extracting only true environmental changes.

[0032] After alignment, the change detection unit 23 analyzes the difference between the two images and detects at least one of the following as a change in the environment: the appearance of a new object, the disappearance of an object, a change in the state of an object (such as a change in shape, color, position, or orientation), or a change in the scene structure (such as the opening or closing of a passage) (S703).

[0033] Various image processing and machine learning techniques can be used to detect these changes (S703). For example, by using background subtraction to extract the difference between a pre-built background model (such as data representing typical past situations) and the current image, newly appearing or disappearing objects can be detected. Alternatively, by applying deep learning-based object detection models (e.g., YOLO, SSD, Faster R-CNN) to both the current image and a comparison of past image data, and comparing the list of detected objects (type, location, size, confidence score, etc.), it is possible to capture increases, decreases, or significant changes in objects. Similarly, it is also effective to use semantic segmentation models (e.g., DeepLab, U-Net) to classify each pixel in an image into categories such as roads, sidewalks, buildings, bushes, vehicles, and people, and to detect changes in the composition and arrangement of each region. For example, it is possible to identify changes such as "a part of an area that was previously recognized as a 'sidewalk' is now recognized as an 'obstacle'."

[0034] In particular, the change detection unit 23 detects an object that was not recognized in the comparison past image data but is recognized as newly appearing in the received current image data as a change in the environment, and determines whether this corresponds to a "change from zero to one" (S704). This is a very important type of change for the early detection of new obstacles, abandoned objects, or suspicious structures.

[0035] Furthermore, in order to distinguish whether the detected change is temporary (e.g., passing vehicles or pedestrians, or debris blown by the wind) or somewhat persistent (e.g., abandoned luggage, or installed construction barricades), the change detection unit 23 determines whether the detected environmental change has persisted for a predetermined period of time or longer (S705). If it has persisted for a predetermined period of time or longer, the change is recognized as a valid change (significant change) (S706) and sent to the subsequent risk determination unit. This determination of persistence can be performed, for example, by tracking whether the same change is continuously detected in multiple consecutive image frames. This significantly reduces accidental false positives and allows for focusing on changes that truly warrant attention. Figure 7 is a flowchart showing the sequence of this change detection process.

[0036] (4) Processing of the determination unit 24 The determination unit 24 receives the environmental changes detected by the change detection unit 23 as input (Figure 8: S801), and based on these changes, determines the degree of potential danger to the person being monitored M (see Figure 5: S205, Figure 8).

[0037] The determination unit 24 uses artificial intelligence (AI) to determine the level of risk. This AI is built on, for example, deep learning models (e.g., convolutional neural networks (CNNs), recurrent neural networks (RNNs), Transformers, etc.) or machine learning algorithms such as gradient boosting trees and support vector machines. This AI model is pre-trained using diverse data related to the safety of those being monitored, especially children.

[0038] (a) Image data that takes into account the image characteristics from a child's perspective: Image and video data that mimics the height and field of view of a child, as captured by the camera 105 of the monitored terminal 10. This includes both safe situations in various environments (sunny, rainy, day and night, indoors and outdoors, etc.) and situations where danger is potentially present (e.g., intersections with poor visibility, construction sites, deserted alleys, places where suspicious objects are placed, etc.).

[0039] (b) Case knowledge regarding past accidents and incidents involving children: Data on actual cases of traffic accidents involving children, attempted kidnappings and abductions, falls, etc. From this data, the circumstances (location, time, weather, surrounding environment, perpetrator's behavior patterns, etc.), causes, and consequences of the accidents and incidents will be extracted and used as features for the AI ​​to learn dangerous patterns. Text data, statistical data, or simulation video data that reproduces the situation can also be used for this purpose.

[0040] The AI ​​in the judgment unit 24 comprehensively evaluates multiple factors such as the type, size, location, and duration of the detected environmental change, as well as the surrounding conditions (presence of other objects, time of day, weather, etc.) through this learning data, and outputs the degree of danger to the person being monitored M as a score or category (e.g., low, medium, high) (S802).

[0041] The AI ​​in the judgment unit 24 may be configured to take into account the risk perception characteristics of the person being monitored M according to their age and developmental stage (as part of S802) in order to perform more accurate and personalized risk assessments. For example, the model may incorporate differences in behavioral tendencies and cognitive abilities by age, such as the fact that toddlers tend to act impulsively and are easily distracted by obstacles at low positions, while young elementary school children tend to approach dangerous places out of curiosity. It is also possible to learn specific environmental change patterns in the past travel routes of a particular person being monitored M (for example, that there is a lot of illegal parking on a particular road, but that is normal in that area) and improve the ability to identify truly "unusual" dangerous changes for that person being monitored.

[0042] Of particular importance is that the determination unit 24 is configured to determine whether or not the environmental change detected by the change detection unit 23 as a "change from zero to one" (S803), and if so, to evaluate the degree of danger as relatively higher compared to other types of environmental changes (S804). This is based on the idea that the appearance of new objects that did not exist in the past often poses a high potential danger, such as the installation of suspicious objects or the occurrence of unexpected obstacles, and therefore should be given special attention.

[0043] Furthermore, the determination unit 24 can consider not only changes in the physical environment but also risks caused by human factors. Specifically, it analyzes time-series data of image data and location information obtained from the monitored person's terminal 10 to recognize the behavioral patterns of other people around the monitored person M. It then determines whether a specific, predefined suspicious behavioral pattern (for example, persistently tracking the monitored person M, lying in wait for a long time in a specific location, approaching at an unnatural distance, hiding behind an object and observing, etc.) has been detected (part of S805). If detected, the situation is taken into account in the risk assessment, and the risk level is evaluated or considered as high (S806). This recognition of behavioral patterns utilizes AI-based person detection, tracking, and behavioral recognition technology.

[0044] Furthermore, in addition to the AI-based learning model, the judgment unit 24 can also refer to a list of dangerous objects that children should be particularly careful of (e.g., knives, syringes, medicine containers, exposed electrical wires, etc.) and a database of typical dangerous behavior patterns (e.g., patterns of methods used by kidnappers, etc.) that have been created in advance by experts. The judgment unit 24 determines whether the detected environmental changes (e.g., the type of object identified by image recognition) or observed behavior patterns match or are similar to dangerous elements in these databases (part of S805), and if they match or are similar, the judgment unit 24 rates the degree of danger particularly highly (S806). This reinforces the ability to detect and evaluate known and clear danger factors that cannot be covered by AI learning alone, thereby increasing the reliability of the system. The final danger level determined in this way is output as a score or category (S807). Figure 8 is a flowchart showing an example of the danger level determination process in the judgment unit 24.

[0045] (5) Processing of the transmitting unit 25 The transmitting unit 25 transmits information regarding the change in the environment to the monitor terminal 30 (S207) if the degree of danger determined by the determination unit 24 meets a predetermined standard (Yes in Figure 5: S206), or if a change requiring special attention, such as a "change from zero to one," is detected.

[0046] The notification information transmitted by the transmitting unit 25 includes information that helps the caregiver U quickly and accurately grasp the situation and take necessary actions. Specifically, it includes at least one of the following pieces of information: (a) Location information: Latitude and longitude information, address, or location on a map of the place where the environmental change was detected. This allows the caretaker to accurately determine the geographical location of the dangerous area. (b) Image data: Image data showing the change in the environment. This includes, if possible, the image at the time the change was detected, as well as past image data used for comparison by the change detection unit 23, and it is desirable that the images be presented in a way that allows for a visual comparison of the situation before and after the change. For example, as shown in Figure 9, the image before the change 902a and the image after the change 902b are displayed side by side on the screen 900 of the monitor terminal 30. (c) Type of change: The category of environmental change identified by the determination unit 24. For example, it is displayed in easy-to-understand terms such as "Road construction begins," "New obstacle (e.g., abandoned bicycle)," or "Suspicious person's behavior." In the example in Figure 9, it is displayed as Type of change 904. (d) Level of danger: A level indicating the degree of danger as assessed by the judgment unit 24. For example, this could be expressed as a tiered system such as "Caution," "Warning," or "Danger," a numerical score, or a color-coded system (e.g., green, yellow, red). In the example in Figure 9, it is displayed as a danger level of 903. (e) Explanatory information regarding the basis for the judgment: A brief explanation of why the AI ​​(judgment unit 24) made that judgment regarding the level of danger, or information that highlights specific objects or areas in the image that served as the basis for the judgment. This allows the observer to understand the AI's judgment process to some extent, thereby increasing the reliability of the notification. In the example in Figure 9, this is displayed as Situation Explanation 905.

[0047] Notifications containing this information are received on the caregiver's terminal 30 in various forms, such as push notifications, SMS, and email, and are configured to allow detailed viewing on a dedicated monitoring app. This enables caregivers to quickly obtain important information related to the safety of the person being monitored and to make appropriate decisions and take appropriate actions (for example, contacting the person being monitored, contacting the school, or rushing to the scene if necessary).

[0048] (6) Processing of the image correction unit 28 The server 20 of this embodiment may further include an image correction unit 28 (see Figure 4). The image correction unit 28 applies various image correction processes to the image data received by the receiving unit 21 or the image data used for comparison by the change detection unit 23 in order to reduce the effects of environmental fluctuations and improve image quality.

[0049] For example, as part of illumination change compensation processing, histogram flattening and tone mapping are performed to adjust the brightness and contrast of the image. This can improve the visibility of images taken in dark places and suppress overexposure in overly bright places.

[0050] For HDR (High Dynamic Range) composite processing, the server 20 receives multiple images (so-called exposure bracketed images) taken in succession by the monitored terminal 10 with different exposure settings, and combines them to generate a wide dynamic range image with minimal overexposure and underexposure. Alternatively, an algorithm that simulates an HDR effect from a single image may be used.

[0051] Noise reduction processing involves using spatial filters such as Gaussian filters, median filters, and bilateral filters, as well as temporal filters that utilize information in the time axis direction, or deep learning-based noise reduction models to reduce random noise and sensor noise contained in the image.

[0052] By appropriately applying these image correction processes, the quality of image data captured under diverse outdoor environmental conditions can be stabilized, improving the accuracy and reliability of image recognition and analysis processing in the subsequent change detection unit 23 and judgment unit 24. As a result, the overall robustness of the system is enhanced, and stable hazard detection performance can be expected under various conditions.

[0053] (7) Processing of the action detection unit 26 The server 20 of this embodiment may further include a behavior detection unit 26 (see Figure 4). The behavior detection unit 26 acquires sensor data (time-series data from acceleration sensors, gyro sensors, geomagnetic sensors, etc.) transmitted from the IMU 107 of the monitored person terminal 10 via the receiving unit 21, and detects unusual behavior or changes in the state of the monitored person M based on this data.

[0054] Unusual behaviors could include, for example, detecting a "fall" from a sudden change in accelerometer readings or a specific pattern, detecting a "sudden stop" or "sudden acceleration (such as running)" from GPS location and acceleration data, or detecting unusually fast movement or, conversely, staying in the same place for an extended period. Techniques such as thresholding, pattern recognition, and machine learning-based behavior classification can be used for these detections.

[0055] When the behavior detection unit 26 detects unusual behavior of the person being monitored M, the determination unit 24 operates to evaluate the degree of danger related to environmental changes before and after the detection point at a higher level than usual. For example, if an obstacle is found in the surrounding image immediately after a fall is detected, or if a new construction site appears immediately before the place where the person made a sudden stop, the determination unit 24 will judge the danger level to be higher. In this way, by taking into account contextual information such as the behavioral changes of the person being monitored, the accuracy and urgency of the assessment of the danger level of environmental changes can be improved, making it possible to generate alerts that are more appropriate to the situation.

[0056] (8) Hierarchical processing in the change detection unit 23 The change detection unit 23 can employ a hierarchical processing structure to improve processing efficiency and real-time performance.

[0057] As the first step, a relatively computationally inexpensive and simple algorithm is used to quickly screen for candidate regions that may show a rough change between the input current image data and past image data (or data representing a typical past situation). Possible simple algorithms for this include, for example, dividing the image into multiple blocks and comparing simple features such as the average brightness, color histogram, and edge density of each block, or downsampling the image and then calculating the difference.

[0058] In the second stage, a more computationally intensive but detailed analysis algorithm is applied only to the candidate change regions extracted in the first stage. This includes object detection, semantic segmentation, detailed feature point matching, and geometric consistency verification.

[0059] This hierarchical processing approach avoids performing computationally intensive analysis on the entire image at all times, allowing computing resources to be concentrated on areas with a high probability of change. As a result, the processing load on server 20 is significantly reduced, and its ability to process data from a large number of monitored individuals in near real-time is improved. This increases system responsiveness and enables faster danger notifications.

[0060] (9) Generation and provision of risk map by map processing unit 27 The server 20 of this embodiment may further include a map processing unit 27 (see Figure 4). Based on the concept of crowdsourcing, the map processing unit 27 generates and updates a local risk map by utilizing information collected from multiple monitored terminals 10.

[0061] Specifically, the map processing unit 27 receives information transmitted from individual monitored terminals 10 via the receiving unit 21 regarding environmental changes detected by the change detection unit 23 (such as the type of change, location, and time of occurrence), and information regarding the degree of danger determined by the determination unit 24. At this time, information that could identify an individual is removed or generalized by the anonymization processing unit 29, which will be described later.

[0062] The collected data is stored in the database of the cloud server 20C, and the map processing unit 27 statistically analyzes this data. For example, it aggregates and analyzes what types of environmental changes occur and how frequently they occur for specific geographical areas (e.g., municipalities, school districts, or finer grid cells) or specific routes (e.g., major school routes), and how often they are judged to be dangerous.

[0063] Based on these analysis results, the map processing unit 27 generates hazard map data that visualizes hazard information on a map. The hazard map data is represented in a format such as assigning a score to each area or route segment on the map according to the frequency of past hazard occurrences and the average hazard level, and then color-coding it based on the score (e.g., a gradient from safe green to dangerous red) or placing icons according to the type of hazard (e.g., construction mark, suspicious person warning mark, etc.). This hazard map data is updated whenever new information is collected, or periodically.

[0064] Furthermore, the map processing unit 27 provides the generated or updated hazard map data to the caregiver terminal 30. For example, the caregiver app on the caregiver terminal 30 is provided with a function to view this hazard map on a map. The caregiver U can refer to this map to check for potential hazards around the child's school route or playground, and to use it as a reference when choosing a safer route. It is also possible to proactively provide information, such as pushing a notification of the hazard level of an area when the person being cared for M approaches that area. Figure 10 shows an example screen 1000 of such a hazard map displayed on the caregiver terminal 30, with high-risk areas 1002 and warning icons 1003 displayed on the map 1001.

[0065] This crowdsourced risk mapping function allows for the sharing of broader and more objective risk information that cannot be obtained solely from the experiences of individual users, and is expected to contribute to improving safety awareness throughout the community and to the consideration of preventive measures.

[0066] Figure 13 is a screenshot showing a specific example of the hazard map displayed on the monitor terminal 30. In this example, it shows that the person being monitored A (not shown) has left home 1004 and is moving towards the destination school 1007. On map 1001, the route already taken by person A is shown as a solid line 1005, and the route that person A usually uses to go to school is shown as a dotted line 1006. In addition, along the school route 1006, there are multiple areas 1002 (high) which have been determined to be high-risk based on past data, and warning icons 1003 that indicate points requiring particular attention. By checking this hazard map 1000 (see Figure 10) in real time, the caregiver U can, for example, send a voice message to the monitored person's terminal 10 to warn the monitored person A when they approach a location requiring attention indicated by a warning icon 1003, or check real-time image data transmitted from the monitored person's terminal 10's camera 105 to confirm whether the monitored person A has safely passed through the location. This allows the caregiver to take more proactive and specific actions against potential dangers, contributing to ensuring the safety of the monitored person.

[0067] (10) Privacy protection by the anonymization processing unit 29 and the consent management unit Protecting the privacy of those being monitored and those monitoring them is extremely important when realizing functions such as utilizing other users' data and generating and sharing risk maps. The server 20 of this embodiment is equipped with means for this purpose.

[0068] Server 20 is equipped with an anonymization processing unit 29 (see Figure 4). The anonymization processing unit 29 automatically detects, removes, or masks personally identifiable information from image data received by the receiving unit 21 or from image data included in information aggregated and shared with the cloud server 20C. For example, it detects face regions in images using AI-based facial recognition technology and applies blurring or mosaic processing. Similarly, it detects names and addresses written on nameplates, vehicle license plates, etc., and applies masking processing. Furthermore, GPS location information is generalized (coarsened) to a certain range (for example, a circle with a radius of 50 meters or a specific road section) rather than pinpoint coordinates, making it difficult to identify an individual's detailed behavioral history. These anonymization processes are performed before the data is stored on the cloud server 20C or before it is used as data by other users.

[0069] Furthermore, the server 20 includes a consent management unit (not shown). The consent management unit clearly and specifically explains to the user (primarily the guardian, who is the caregiver) the types of information collected by the system (image data, location information, sensor data, etc.), the purpose of using each piece of information (for self-risk detection, for improving the accuracy of the system, for providing information to other users after anonymization and statistical analysis, for creating a risk map, etc.), and whether and to what extent the information will be provided to third parties, and obtains explicit prior consent (opt-in method) for each item individually.

[0070] The consent management unit provides an interface (for example, the privacy settings screen 1100 shown in Figure 11) through an app on the monitor terminal 30, allowing users to check and easily change their privacy settings at any time. On this screen, users can set items such as "Allow other users to use your data" and "Allow anonymous data provision to the risk map" to on or off. Each functional unit of the server 20 (especially the historical data identification unit 22, the map processing unit 27, the transmission unit 25, etc.) controls the scope of data collection, use, and sharing based on the user's consent status managed by the consent management unit. This makes it possible to provide useful system functions while respecting the user's privacy rights and complying with relevant laws and regulations.

[0071] (Specific examples of device attachment and recording control for the person being monitored) The camera 105 of the monitored terminal 10 is fixed to a dedicated clip or mount, for example, on the shoulder strap of a school bag, chest harness, or brim of a hat, so that it can stably capture images in the direction of travel from a position close to the eye level of the monitored person M. Alternatively, a clip structure or mounting attachment part for direct attachment to clothing or a bag may be integrally molded on the back or side of the housing. Or, when the device is worn around the neck on a strap, it may be attached to the collar using the above-mentioned clip or mount. This provides images that are close to what the child sees, contributing to improved accuracy of AI-based situation recognition and danger assessment. In addition, the IMU 107 may be used to detect the tilt and shaking of the camera, and software-based correction may be performed to stabilize the image, similar to a digital gimbal function.

[0072] In addition to remote control from the monitor terminal 30, recording can also be autonomously controlled from the monitored terminal 10. For example, the GPS receiver 106 can be used to determine the location of the monitored person M, and recording can be automatically started and stopped when the person enters or leaves a pre-set geofence (virtual geographical boundary). Specifically, recording can be stopped in areas where privacy considerations are necessary, such as the grounds of a home or school, and automatically started when the person leaves. Furthermore, it is also effective from a convenience standpoint to allow the monitored person M to control recording with simple voice commands such as "start recording" and "stop recording" via the audio input / output unit 108. These recording controls also contribute to optimizing battery consumption.

[0073] Furthermore, the monitored terminal 10 may be equipped with a power-saving control function that monitors the remaining charge of the built-in battery 109 and dynamically adjusts the frequency of image data acquisition, the frequency of uploading to the server 20, or the resolution, compression ratio, and frame rate of the transmitted image data according to the remaining charge. It is desirable that this power-saving control is not performed uniformly based solely on the remaining battery charge, but rather takes other information into consideration to prioritize the safety of the monitored person M. For example, even if the battery charge is low, if the location information obtained from the GPS receiver 106 is outside the safe zone of the monitored person M (e.g., home, school, or daily commute route) that has been set in advance, or if the person is in an area indicated as high-risk in the risk map data (see Figure 10) provided by the map processing unit 27 (see Figure 4), the frequency of image acquisition and uploading will be maintained at the normal or higher frequency. Conversely, if it is determined that the person is in a safe zone or a low-risk area, the frequency will be more actively reduced. Furthermore, the time of day is also taken into consideration. For example, during times when the risk is generally expected to be higher, such as late at night, early in the morning, or specific times when the person being monitored M is likely to be acting alone, the priority of image acquisition and transmission is increased even if the battery level is low. In addition to these, the monitored terminal 10 may, for example, comprehensively evaluate the distance from the current location acquired by the GPS receiver 106 to a pre-registered destination such as home, the speed of movement estimated from the accelerometer (part of the IMU 107) and GPS information, and the current battery level, and adaptively control the image resolution, frame rate, or upload frequency (for example, switching from real-time upload to batch upload, or prioritizing the transmission of location information only) so that at least the surrounding circumstances and route of the person being monitored M are continuously recorded until they reach their destination.This control system utilizes location and sensor information for the clear purpose of ensuring the safety of the person being monitored. For example, if the destination is far away and the battery level is low, it will gradually reduce image resolution to maximize recording time and transmission of location information. Conversely, if the destination is close, it will maintain the recording quality of the final critical section as much as possible. This is fundamentally different from the battery-saving measures used when watching videos on general electronic devices or simply reducing data usage. It is a meticulous power management strategy specifically tailored for monitoring applications. In this way, by comprehensively evaluating multiple pieces of information such as battery level, the relationship between current location and range of activity / risk map, distance to home and speed of movement, and time of day, and optimizing the image acquisition and transmission policy according to the situation, it is possible to reduce battery consumption while increasing the continuity of information gathering to maximize the safety of the person being monitored M.

[0074] (Specific examples of continuous learning and personalization of AI models) The AI ​​model used in the judgment unit 24 of server 20 is not fixed after being trained once, but should be continuously improved and evolved. To this end, server 20 may be equipped with a learning mechanism (not shown) that periodically retrains and updates the AI ​​model by utilizing various data collected from the entire system.

[0075] This learning mechanism utilizes, for example, image data, location information, detected environmental change patterns, the AI's initial judgments on these changes, and, importantly, feedback from the monitors (e.g., evaluations such as "this notification was actually dangerous" or "this was a false alarm," as well as information on the actions taken). By anonymizing and statistically analyzing this information and using it as training data to retrain the AI ​​model, the accuracy and comprehensiveness of danger assessment can be continuously improved.

[0076] Furthermore, it is conceivable to perform personalization by learning the behavioral patterns, daily travel routes, history of dangerous situations encountered in the past, and feedback tendencies from caregiver U of a specific person being monitored, and then adjusting the parameters of the AI ​​model specifically for that person or optimizing the threshold for changes that warrant attention. For example, if a child frequently plays in a particular park, it becomes possible to individually optimize the system by setting a higher sensitivity to unusual changes within that park (such as the appearance of an unfamiliar person) than in other locations. This enables more detailed and effective monitoring tailored to the individual child's situation, rather than uniform judgments.

[0077] Although embodiments of the present invention have been described in detail above, these are illustrative and do not limit the scope of the present invention. It is possible to add, omit, substitute, or otherwise modify the components without departing from the spirit of the present invention. For example, in the above embodiments, the information processing device was used as a server, but it is also possible to distribute some or all of its functions to the monitored person's terminal or the monitor's terminal. Furthermore, the specific AI algorithm, learning method, and numerical values ​​of various parameters are not limited to those shown herein.

[0078] [General tasks] In technologies that support the safety of people on the move, especially children and the elderly, there is a need for a means to continuously monitor changes in the environment along the person's movement route and effectively notify caregivers of potential dangers when those changes suggest them. Conventional technologies have not been able to provide sufficient support in terms of automatically detecting subtle environmental changes that may occur on a daily basis, determining what kind of dangers they pose to the person, and proactively providing specific warnings to caregivers. Therefore, the challenge is to provide a more advanced danger notification system that can contribute to reducing the burden on caregivers and improving their sense of security.

[0079] [Note 1] [Issues corresponding to Appendix 1] To support the safety of the person being monitored and reduce the burden on the monitor by automatically detecting environmental changes along the person's movement path and notifying the monitor of potential dangers. [Contents of Appendix 1] Information processing device comprising: a receiving unit that receives image data and location information transmitted from a monitoring terminal equipped with a camera; a past data identification unit that identifies comparison past image data, which is past image data along the monitoring person's movement path, based on the received location information; a change detection unit that compares the received image data with the identified comparison past image data and detects changes in the environment around the movement path based on the difference between the two; a determination unit that determines the degree of potential danger to the monitoring person based on the detected environmental changes; and a transmission unit that transmits information regarding the environmental changes to the monitoring terminal based on the determined degree of danger. [Effects of Appendix 1] According to the above configuration, it becomes possible to detect potential dangers caused by changes in the environment around the path the person being monitored travels at an early stage and to appropriately notify them of information regarding such dangers. As a result, the caregiver can receive support to more effectively ensure the safety of the person being monitored, reducing the burden of daily safety checks and increasing their sense of security.

[0080] [Note 2] [Issues corresponding to Appendix 2] To improve the reliability of change detection by identifying historical image data used for comparison as appropriate and spatially corresponding to the current situation when detecting environmental changes. [Contents of Appendix 2] The information processing apparatus according to Appendix 1, characterized in that the past data identification unit identifies image data of the same location or a nearby location in the past along the movement route of the person being monitored, corresponding to the received current location information, as the past image data for comparison. [Effects of Appendix 2] According to the above configuration, by using historical image data highly relevant to the current location as a comparison target, the accuracy of detecting environmental changes can be improved.

[0081] [Note 3] [Issues corresponding to Appendix 3] In detecting environmental changes, considering everyday behavioral patterns and short-term environmental stability reduces false positives and helps capture more meaningful "unusual" changes. [Contents of Appendix 3] The information processing apparatus according to Appendix 2, characterized in that the past data identification unit selects the past image data for comparison, taking into consideration the periodicity of the person being monitored's behavior or the stability of the recent environment. [Effects of Appendix 3] According to the above configuration, by taking into account the behavioral rhythm of the person being monitored and temporary, ongoing environmental conditions such as construction work, the accuracy of change detection can be improved and unnecessary warnings can be reduced.

[0082] [Note 4] [Issues corresponding to Appendix 4] The goal is to generate data that captures representative conditions of a given location from multiple past images, rather than a single past image, thereby improving the stability of the comparison criteria and enhancing noise tolerance for change detection. [Contents of Appendix 4] The information processing apparatus according to Appendix 3, characterized in that the past data identification unit generates data representing a typical past situation at a given location from image data of multiple past points in time, and identifies this data as the comparative past image data. [Effects of Appendix 4] According to the above configuration, the reliability of detecting environmental changes can be improved by using a stable comparison standard that is less susceptible to temporary fluctuations.

[0083] [Note 5] [Issues corresponding to Appendix 5] This involves generating data representing typical past situations through statistical processing of image features, and constructing comparative criteria that capture the structural characteristics of the scene. [Contents of Appendix 5] The information processing apparatus according to Appendix 4, characterized in that the data representing typical past situations is generated by statistically processing image features extracted from image data from multiple past points in time. [Effects of Appendix 5] According to the above configuration, it becomes possible to generate representative situations based on a deeper understanding of the image content, and to detect more meaningful environmental changes.

[0084] [Note 6] [Issues corresponding to Appendix 6] To generate data representing typical past situations by selecting and combining representative images, thereby constructing a visually clear and noise-reduced comparison standard. [Contents of Appendix 6] The information processing apparatus according to Appendix 4, characterized in that the data showing representative past conditions is generated by selecting or combining one or more representative images from the multiple past image data points. [Effects of Appendix 6] With the above configuration, the accuracy of change detection can be improved by using an image that is closer to normal conditions, with the effects of temporary objects and noise removed, as a comparison standard.

[0085] [Note 7] [Issues corresponding to Appendix 7] Even when the monitored individual has limited historical data, the system can utilize anonymized data collected from other users to provide environmental change detection capabilities, even in the initial stages of system use or in unfamiliar locations. [Contents of Appendix 7] The information processing device according to Appendix 1, characterized in that, when there is no past image data of the person being monitored in the monitoring person's travel route, or when the past image data does not meet a predetermined standard, it identifies the comparison past image data from past image data transmitted from another monitoring person's terminal, which has been anonymized in advance and stored on a cloud server, based on the received current location information and direction of travel. [Effects of Appendix 7] With the above configuration, even immediately after starting to use the system or when visiting a new location, it becomes possible to detect and compare environmental changes using crowdsourced data, thereby improving the convenience of the service.

[0086] [Note 8] [Issues corresponding to Appendix 8] When comparing current and past image data, this involves correcting for apparent differences due to variations in shooting location and orientation, and accurately extracting only the true environmental changes. [Contents of Appendix 8] The information processing apparatus according to Appendix 1, characterized in that the change detection unit performs a high-precision alignment process to associate the shooting position and shooting direction of the current image data and the past image data for comparison between the two, and then performs the comparison. [Effects of Appendix 8] With the above configuration, by correcting camera movement and viewpoint shifts, false detections of environmental changes can be reduced, and detection accuracy can be improved.

[0087] [Note 9] [Issues corresponding to Appendix 9] To specifically detect environmental changes as the appearance, disappearance, or state changes of objects, or as changes in scene structure, and to clearly understand what kind of changes have occurred. [Contents of Appendix 9] The information processing apparatus according to Appendix 8, characterized in that the change detection unit detects at least one of the following as a change in the environment: the appearance of a new object, the disappearance of an object, a change in the state of an object, or a change in the scene structure. [Effects of Appendix 9] With the above configuration, by specifically classifying and identifying the content of the detected changes, subsequent risk assessments and information provision to caregivers can be performed more appropriately.

[0088] [Note 10] [Issues corresponding to Appendix 10] The key is to specifically detect "zero-to-one changes"—the emergence of objects that did not previously exist—which are particularly noteworthy among environmental changes, and to expedite responses to potential hazards. [Contents of Appendix 10] The information processing apparatus according to Appendix 9, characterized in that the change detection unit specifically detects, as a change in the environment, an object that was not recognized in the comparative past image data but is recognized as newly appearing in the received current image data, as a "change from zero to one". [Effects of Appendix 10] According to the above configuration, changes requiring particular attention, such as the appearance of suspicious objects or new obstacles, can be identified early, leading to rapid risk assessment and notification.

[0089] [Note 11] [Issues corresponding to Appendix 11] To reduce false alarms and improve the reliability of notifications by distinguishing between temporary and persistent environmental changes and treating only truly noteworthy persistent changes as valid changes. [Contents of Appendix 11] The information processing apparatus according to Appendix 9, characterized in that the change detection unit treats the detected change in the environment as valid if it persists for a predetermined time or longer. [Effects of Appendix 11] According to the above configuration, by excluding temporary changes such as passing vehicles and focusing on persistent changes such as abandoned objects and construction barricades, more important information can be provided to observers.

[0090] [Note 12] [Issues corresponding to Appendix 12] Using artificial intelligence, the system learns about children's perspective characteristics and past accident and incident cases to comprehensively determine the degree of danger to the person being monitored when a change in the environment is detected. [Contents of Appendix 12] The information processing device according to Appendix 1, characterized in that the determination unit uses artificial intelligence to determine the degree of danger based on the image characteristics from the perspective of the person being watched and the results of learning past case knowledge regarding accidents or incidents involving the person being watched or other children. [Effects of Appendix 12] The above configuration enables sophisticated risk assessment based on children's unique situational awareness and past knowledge, allowing for more accurate warnings.

[0091] [Note 13] [Issues corresponding to Appendix 13] In risk assessment, individual factors such as the age, developmental stage, and past movement patterns of the person being monitored should be taken into consideration to conduct a more personalized and appropriate risk assessment. [Contents of Appendix 13] The information processing device according to Appendix 12, characterized in that the determination unit is configured such that the artificial intelligence determines the degree of danger by considering the danger perception characteristics of the person being monitored according to their age and developmental stage, or specific environmental change patterns in the person being monitored's past travel routes. [Effects of Appendix 13] According to the above configuration, it becomes possible to make detailed risk assessments tailored to the individual characteristics of the person being monitored, enabling more practical and non-uniform monitoring.

[0092] [Note 14] [Issues corresponding to Appendix 14] This involves evaluating the risk of a "zero-to-one change"—the appearance of a new object—as relatively higher than other changes, and drawing attention to changes with a high potential risk. [Contents of Appendix 14] The information processing apparatus according to Appendix 12, characterized in that the determination unit is configured to evaluate the degree of danger as relatively higher when the detected change in the environment is a "change from zero to one" compared with other types of changes in the environment. [Effects of Appendix 14] The above configuration allows for increased sensitivity to changes that require particular attention, such as new obstacles or suspicious objects, and enables appropriate prioritization of warnings to the observer.

[0093] [Note 15] [Issues corresponding to Appendix 15] This involves detecting not only changes in the physical environment but also suspicious behavioral patterns of other individuals around the person being monitored, and conducting a comprehensive risk assessment that takes into account human-induced risks. [Contents of Appendix 15] The information processing device according to Appendix 12, characterized in that the determination unit analyzes the behavioral patterns of other people in the vicinity of the person being monitored in a time series, and when a predetermined specific behavioral pattern is detected, it takes that situation into consideration when determining the degree of danger. [Effects of Appendix 15] According to the above configuration, incorporating human threats such as tracking and ambushes into the risk assessment enables more multifaceted support for ensuring safety.

[0094] [Note 16] [Issues corresponding to Appendix 16] In addition to AI-based learning models, this approach involves referencing databases of hazardous materials and risky behavior patterns defined by experts to enhance the ability to detect and assess known and clearly defined risk factors. [Contents of Appendix 16] The information processing device according to Appendix 12, characterized in that the determination unit further refers to a database of pre-registered lists of dangerous objects or dangerous behavior patterns that require particular attention from children, and evaluates the degree of danger particularly highly when the detected change in the environment matches or is similar to these. [Effects of Appendix 16] The above configuration improves the accuracy of evaluations of known risk factors that cannot be fully covered by AI learning alone, thereby enhancing the reliability of the system.

[0095] [Note 17] [Issues corresponding to Appendix 17] To provide caregivers with specific and easy-to-understand information, such as the location where danger was detected, images indicating the change, the type of change, the level of danger, and the basis for the judgment, in order to support a quick and appropriate response. [Contents of Appendix 17] The information processing apparatus according to Appendix 1, characterized in that the transmitting unit transmits, as information relating to the change in the environment, at least one of the location information where the change in the environment was detected, image data indicating the change in the environment, the type of change in the environment, the level of the determined degree of danger, and explanatory information relating to the basis for determining the degree of danger. [Effects of Appendix 17] According to the above configuration, the caregiver will be able to grasp the situation from multiple perspectives and make more accurate judgments and responses.

[0096] [Note 18] [Issues corresponding to Appendix 18] The system applies image correction processing, such as illuminance change compensation, HDR synthesis, and noise reduction, to the received image data and the image data used for comparison, thereby reducing the impact of environmental changes and improving the accuracy of change detection and risk assessment. [Contents of Appendix 18] The information processing apparatus according to Appendix 1, further comprising an image correction unit that performs at least one of illuminance change compensation processing, HDR (high dynamic range) synthesis processing, or noise reduction processing on image data received by the receiving unit or image data used for comparison by the change detection unit. [Effects of Appendix 18] The above configuration allows for stable image quality under various shooting conditions, improves the accuracy and reliability of subsequent image recognition processing, and enhances the overall robustness of the system.

[0097] [Note 19] [Issues corresponding to Appendix 19] The system detects unusual behaviors of the person being monitored, such as falls or sudden stops, from sensor data, and then correlates these behaviors with environmental changes to assess the level of risk, thereby enabling more situation-appropriate risk assessment. [Contents of Appendix 19] The information processing device further includes an action detection unit that receives sensor data indicating the movements of the person being monitored from the terminal for the person being monitored, and detects unusual behavior of the person being monitored, such as falling, sudden stopping, or moving at an unusual speed, based on the sensor data, and the determination unit, when the unusual behavior is detected, evaluates the degree of danger of the change in the environment before and after the detection time to be higher than under normal circumstances, as described in Appendix 1. [Effects of Appendix 19] According to the above configuration, changes in the behavior of the person being monitored can be used as a trigger to adjust the urgency and importance of the risk assessment of environmental changes, enabling the generation of more accurate alerts.

[0098] [Note 20] [Issues corresponding to Appendix 20] In change detection processing, a hierarchical processing approach is employed, which first narrows down a wide range of candidates using a simple algorithm, and then performs a detailed analysis, thereby improving processing efficiency and real-time performance. [Contents of Appendix 20] The information processing apparatus according to Appendix 1, characterized in that the change detection unit first extracts a wide range of candidate change regions using a simple algorithm, and then performs hierarchical processing by applying a more detailed analysis algorithm to the candidate change regions. [Effects of Appendix 20] With the above configuration, computing resources can be used efficiently, reducing the server's processing load, allowing for near real-time processing of large amounts of data and enabling rapid danger notification.

[0099] [Note 21] [Issues corresponding to Appendix 21] To provide more comprehensive safety information by collecting environmental change information and risk assessment information from multiple individuals being monitored, anonymizing and statistically processing it, and then aggregating it to generate and update risk maps that show risk trends in specific areas and routes. [Contents of Appendix 21] The information processing device described in Appendix 1 is characterized in that it aggregates information on environmental changes and determined risk levels received from multiple monitoring terminals into a cloud server after anonymizing or statistically processing the information so that individuals cannot be identified, generates risk map data showing the trend of environmental changes or the relative risk level in a specific geographic area or route based on this aggregated information, and further includes a map processing unit that updates the risk map data in accordance with the passage of time or the aggregation of new information. [Effects of Appendix 21] According to the above configuration, crowdsourcing can be used to share wide-area risk information that individual users alone could not obtain, thereby contributing to improving safety awareness throughout the region.

[0100] [Note 22] [Issues corresponding to Appendix 22] To support preventative safety measures by providing generated and updated hazard maps to monitoring devices, which can be used for selecting school routes and pre-checking dangerous areas. [Contents of Appendix 22] The information processing device according to Appendix 21, characterized in that the map processing unit provides the generated or updated risk map data to the monitor terminal in response to a request from the monitor terminal, or in association with the current location or planned route of the person being monitored. [Effects due to Appendix 22] According to the above configuration, the caregiver can develop a safer action plan based on objective risk information, which helps ensure the safety of the person being cared for.

[0101] [Note 23] [Issues corresponding to Appendix 23] The system protects user privacy by automatically detecting and masking or generalizing personally identifiable information, such as faces, nameplates, and license plates, contained in image data collected and used by the system. [Contents of Appendix 23] The information processing device according to Appendix 1, further comprising an anonymization processing unit that automatically detects personally identifiable information such as faces, nameplates, and vehicle license plates in image data received by the receiving unit or image data included in the information aggregated on the cloud server, and applies masking or generalization processing to it. [Effects due to Appendix 23] According to the above configuration, data can be utilized in a way that respects privacy, thereby increasing user confidence and contributing to compliance with laws and regulations concerning the protection of personal information.

[0102] [Note 24] [Issues corresponding to Appendix 24] By obtaining explicit prior consent from users regarding the types of information collected, the purpose of use, and the scope of sharing, and by operating the system based on that consent, we respect users' privacy rights and provide a reliable service. [Contents of Appendix 24] The information processing device according to Appendix 1, further comprising a consent management unit that obtains prior consent from the user regarding the type of information to be collected, its purpose of use, and the scope of sharing, via the monitored person's terminal or the monitor's terminal, and executes or controls at least a part of the processing of each of the above units based on the scope of that consent. [Effects of Appendix 24] The above configuration enables highly transparent information handling and provides an environment where users can use the system with peace of mind. [Explanation of Symbols]

[0103] 1…Risk level notification system 10… Terminal for the person being monitored 11…Camera 12…GPS receiver 13…IMU (Inertial Measurement Unit) 14...Storage section 15…Communication module 16…Processor (device for the person being monitored) 20… Server (information processing device) 20C…Cloud Server 21... Receiver 22…Past Data Identification Section 23... Change detection unit 24…Judgment section 25...Transmitter 26... Behavior detection unit 27…Map Processing Unit 28…Image Correction Section 29…Anonymization Processing Unit 30…Monitoring device 40…Network 101…Processor (terminal for the person being monitored) 102...Memory (device for the person being monitored) 103...Memory Unit (Terminal for the person being monitored) 104...Communication module (terminal for the person being monitored) 105...Camera (device for monitoring the person being monitored) 106…GPS receiver (device for the person being monitored) 107...IMU (Infrared Measurement Unit) 108...Audio input / output unit (terminal for the person being monitored) 109...Battery (for the person being monitored) 201…Processor (Server) 202...Memory (Server) 203...Storage device (server) 204...Communication interface (server) 205… Input / Output Interface (Server) 206... Bus (server) 301…Image and location information database 900…Notification information display screen example 901... Map 902a...Image before change 902b...Image after change 903... Danger level indicator 904...Display of the type of change 905...Situation description display 1000...Example of a danger level map display screen 1001...Map 1002... Danger level area display 1003... Danger icon 1004... Home 1005... Route already traveled 1006...Normal school route 1007…School (destination) 1100...Example of privacy settings screen M... Person being watched over U...Guardian

Claims

1. A receiving unit that receives image data and location information transmitted from a monitoring terminal equipped with a camera, A past data identification unit identifies past image data for comparison, which is past image data of the movement route of the person being monitored, based on the received location information. A change detection unit compares the received image data with the identified past image data for comparison and detects changes in the environment surrounding the travel path based on the difference between the two. A determination unit that determines the degree of potential danger to the person being monitored based on the detected environmental changes, A transmitting unit transmits information regarding the environmental changes to a monitoring terminal based on the determined degree of danger. An information processing device having

2. The information processing apparatus according to claim 1, characterized in that the past data identification unit identifies image data of the same location or a nearby location in the past along the movement route of the person being monitored, corresponding to the received current location information, as the past image data for comparison.

3. The information processing apparatus according to claim 2, characterized in that the past data identification unit selects the past image data for comparison, taking into consideration the periodicity of the person being monitored or the stability of the recent environment.

4. The information processing apparatus according to claim 3, characterized in that the past data identification unit generates data representing a typical past situation at the location from image data of multiple past points in time, and identifies this as the comparative past image data.

5. The information processing apparatus according to claim 4, characterized in that the data showing representative past conditions is generated by statistically processing image features extracted from the image data of multiple past points in time.

6. The information processing apparatus according to claim 4, characterized in that the data showing representative past situations is generated by selecting or combining one or more representative images from the multiple past image data points.

7. The information processing apparatus according to claim 1, characterized in that the past data identification unit identifies the comparison past image data from past image data transmitted from other monitored terminals, which have been anonymized in advance and stored on a cloud server, based on the received current location information and direction of travel, when there is no past image data of the monitored person themselves along the monitored person's movement route, or the predetermined criteria are not met.

8. The information processing apparatus according to claim 1, characterized in that the change detection unit performs a high-precision alignment process to associate the shooting position and shooting direction of the current image data and the past image data for comparison between them, and then performs the comparison.

9. The information processing apparatus according to claim 8, characterized in that the change detection unit detects at least one of the following as a change in the environment: the appearance of a new object, the disappearance of an object, a change in the state of an object, or a change in the scene structure.

10. The information processing apparatus according to claim 9, characterized in that the change detection unit specifically detects, as a change in the environment, an object that was not recognized in the past comparison image data but is recognized as newly appearing in the received current image data as a "change from zero to one".

11. The information processing apparatus according to claim 9, characterized in that the change detection unit treats the detected change in the environment as valid if it persists for a predetermined time or longer.

12. The information processing device according to claim 1, characterized in that the determination unit uses artificial intelligence to determine the degree of danger based on the image characteristics from the perspective of the person being watched and the results of learning past case knowledge regarding accidents or incidents involving the person being watched or other children.

13. The information processing device according to claim 12, characterized in that the determination unit is configured such that the artificial intelligence determines the degree of danger by considering the danger perception characteristics of the person being monitored according to their age and developmental stage, or a unique pattern of environmental changes in the person being monitored's past travel routes.

14. The information processing apparatus according to claim 12, characterized in that the determination unit is configured to evaluate the degree of danger as relatively higher when the detected change in the environment is a "change from zero to one" compared with other types of changes in the environment.

15. The information processing device according to claim 12, characterized in that the determination unit analyzes the behavioral patterns of other people in the vicinity of the person being monitored in a time series, and when a predetermined specific behavioral pattern is detected, it takes that situation into consideration when determining the degree of danger.

16. The information processing device according to claim 12, wherein the determination unit further refers to a database of pre-registered lists of dangerous objects or dangerous behavior patterns that require particular attention from children, and evaluates the degree of danger particularly highly when the detected change in the environment matches or is similar to these.

17. The information processing apparatus according to claim 1, characterized in that the transmitting unit transmits, as information relating to the change in the environment, at least one of the location information where the change in the environment was detected, image data indicating the change in the environment, the type of change in the environment, the level of the determined degree of danger, and explanatory information relating to the basis for determining the degree of danger.

18. The information processing apparatus according to claim 1, further comprising an image correction unit that performs at least one of illuminance change compensation processing, HDR (high dynamic range) synthesis processing, or noise reduction processing on image data received by the receiving unit or image data used for comparison by the change detection unit.

19. The information processing device further includes an action detection unit that receives sensor data indicating the movements of the person being monitored from the terminal for the person being monitored, and detects unusual behavior of the person being monitored, such as falling, sudden stopping, or moving at an unusual speed, based on the sensor data, and the determination unit, when the unusual behavior is detected, evaluates the degree of danger of the change in the environment before and after the detection time to be higher than under normal circumstances, as described in claim 1.

20. The information processing apparatus according to claim 1, characterized in that the change detection unit first extracts a wide range of candidate change regions using a simple algorithm, and then performs hierarchical processing by applying a more detailed analysis algorithm to the candidate change regions.

21. The information processing device according to claim 1, further comprising: an information processing device that aggregates information on environmental changes and determined risk levels received from multiple monitoring terminals into a cloud server after anonymizing or statistically processing the information so that individuals cannot be identified; generates risk map data showing the trend of environmental changes or the relative risk level in a specific geographic area or route based on this aggregated information; and updates the risk map data in accordance with the passage of time or the aggregation of new information.

22. The information processing device according to claim 21, characterized in that the map processing unit provides the generated or updated risk map data to the monitor terminal in response to a request from the monitor terminal, or in association with the current location or planned route of the person being monitored.

23. The information processing device according to claim 1, further comprising an anonymization processing unit that automatically detects personally identifiable information such as faces, nameplates, and vehicle license plates in image data received by the receiving unit or image data included in the information aggregated on the cloud server, and applies masking or generalization processing to it.

24. The information processing device according to claim 1, further comprising a consent management unit that obtains prior consent from the user regarding the type of information to be collected, its purpose of use, and the scope of sharing, via the monitored person's terminal or the monitor's terminal, and executes or controls at least a part of the processing of each of the above units based on the scope of that consent.

25. The steps include receiving image data and location information transmitted from a monitoring terminal equipped with a camera, Based on the received location information, the step of identifying past image data for comparison, which is past image data of the movement route of the person being monitored, The steps include comparing the received image data with the identified past image data for comparison and detecting changes in the environment surrounding the travel path based on the difference between the two, The steps include determining the degree of potential danger to the person being monitored based on the detected environmental changes, Based on the determined degree of danger, the step of transmitting information regarding the change in the environment to the monitor's terminal, An information processing method having

26. Computers, A receiving unit that receives image data and location information transmitted from a monitoring terminal equipped with a camera. A past data identification unit identifies past image data for comparison, which is past image data along the movement path of the person being monitored, based on the received location information. A change detection unit compares the received image data with the identified past image data for comparison and detects changes in the environment surrounding the travel path based on the difference between the two. A determination unit that determines the degree of potential danger to the person being monitored based on the detected environmental changes, and A transmission unit transmits information regarding the environmental changes to a monitoring terminal based on the determined degree of danger. A program designed to function as such.

27. A monitoring device equipped with a camera, A monitoring device and The aforementioned monitoring terminal and a server connected to the monitoring terminal via a network, Equipped with, The aforementioned server, A receiving unit that receives image data and location information transmitted from the aforementioned terminal for the person being monitored, A past data identification unit identifies past image data for comparison, which is past image data of the movement route of the person being monitored, based on the received location information. A change detection unit compares the received image data with the identified past image data for comparison and detects changes in the environment surrounding the travel path based on the difference between the two. A determination unit that determines the degree of potential danger to the person being monitored based on the detected environmental changes, A transmitting unit transmits information regarding the change in the environment to the monitoring terminal based on the determined degree of danger, An information processing system having [a certain feature].

Images