Hazard prediction device, hazard prediction system, hazard prediction method, and hazard prediction program

Abstract

To provide a hazard prediction device, hazard prediction system, hazard prediction method, and hazard prediction program that can effectively and efficiently predict hazards in designated locations such as daycare centers. [Solution] A hazard prediction device for predicting hazards in a predetermined location, comprising: an information analysis unit that acquires image information captured from a predetermined location and outputs detection information which is information about detected objects detected from the image information; and a hazard prediction unit that outputs hazard information which is information about hazards predicted in a predetermined location, generated based on the detection information using basic hazard information collected in advance as hazards predicted in a predetermined location.

Description

Technical Field

[0001] The present invention relates to a technique for predicting dangers in predetermined places such as nurseries.

Background Art

[0002] In facilities where children gather, such as nurseries and kindergartens, various dangers are hidden that arise from children's behaviors and unforeseen situations. In particular, children may take unexpected actions, and facilities where children gather harbor more dangers than general facilities. Some of such dangers can lead to serious accidents, and in such facilities, adults such as nursery teachers usually watch over children to prevent accidents from occurring. However, it is difficult to watch all children at all times, and adults may not notice the hidden dangers.

[0003] Under such circumstances, there is a demand for a technique that enables prediction of dangers lurking in facilities such as nurseries so that adults can respond appropriately. Danger prediction techniques are also required in workplaces and work sites to ensure the safety of staff and workers, and are also required in homes and facilities where elderly people reside to prevent dangerous behaviors of the elderly.

[0004] Many danger prediction techniques have been proposed that predict dangers using image data such as still images and moving videos obtained by imaging facilities or the like that are the targets of danger prediction. For example, Japanese Unexamined Patent Application Publication No. 2021-33404 (Patent Document 1) proposes a danger prediction activity support system that analyzes information of an image captured by a terminal device to detect an object, first extracts pre-set high-priority danger information corresponding to the object, then recalculates the priority based on worker information, and extracts high-priority danger information recalculated from the remaining risk danger information. In the system described in Patent Document 1, a plurality of pieces of danger information are set in advance for each object.

[0005] Japanese Patent Publication No. 2023-79291 (Patent Document 2) proposes a technology that identifies the behavioral patterns of people captured in an image, acquires characteristic information indicating the characteristics of the imaging location, determines the degree of danger that may befall a person using the identified behavioral patterns and acquired characteristic information, and outputs information regarding the determined degree of danger. Patent Document 2 also provides an embodiment of a system that uses the behavioral patterns of children captured in an image and characteristic information of the imaging location to estimate the degree of danger (risk level) indicating the likelihood of a child engaging in dangerous behavior, and notifies the child's chaperone or other person of the estimation result.

[0006] Japanese Patent Publication No. 2021-76934 (Patent Document 3) proposes a technology to prevent dangerous human behaviors such as falls, tripping, and staggering by detecting dangerous situations based on area information such as image information of a predetermined area and detection information used to detect dangerous situations in a predetermined area, and further estimating the cause of the detected dangerous situation. In the technology described in Patent Document 3, image information, distance information, human skeletal information, etc. are used as area information, and information related to dangerous human behavior (dangerous behavior information), information related to the environment of a predetermined area corresponding to the cause of dangerous human behavior (dangerous environment information), information related to the movement of a person in a safe state (safe behavior information), information related to an object in a safe state (safe object information), etc. are used as detection information. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2021-33404 [Patent Document 2] Japanese Patent Publication No. 2023-79291 [Patent Document 3] Japanese Patent Publication No. 2021-76934 [Overview of the Initiative] [Problems that the invention aims to solve]

[0008] However, in the system described in Patent Document 1, since hazard information is set for each object, it may be difficult to accurately predict hazards that may arise from the interaction of multiple objects. Furthermore, while Patent Document 1 states that hazard information includes information on hazards that may occur in an object and information on past incidents involving an object, it does not describe the specific content or method of creating the former information.

[0009] The technology described in Patent Document 2 requires the acquisition of a large amount of information, such as a child's facial image, height, location within the park, direction of movement, speed of movement, type of game being played, whether the child is aware of their surroundings, and personality, in order to determine the degree of danger a person may face, for example, the likelihood of a child engaging in dangerous behavior while playing in a park. Therefore, the scale of the equipment and other devices required to implement this technology may be large, and the processing load may also be large. If the information is insufficient or its accuracy is not sufficient, it may not be possible to accurately determine the degree of danger.

[0010] The technology described in Patent Document 3 requires the use of a large amount of information, such as image information, distance information, human skeletal information, dangerous behavior information, dangerous environment information, safe behavior information, and safe object information, in order to detect dangerous situations. Therefore, it has the same problems as the technology described in Patent Document 2 in terms of device size, processing volume, detection accuracy, etc.

[0011] The present invention has been made in accordance with the circumstances described above, and the object of the present invention is to provide a hazard prediction device, a hazard prediction system, a hazard prediction method, and a hazard prediction program that can effectively and efficiently predict hazards in designated places such as nurseries. [Means for solving the problem]

[0012] The present invention relates to a hazard prediction device for predicting hazards in a predetermined location, and the above object of the present invention is achieved by comprising: an information analysis unit that acquires image information captured from the predetermined location and outputs detection information which is information about detected objects detected from the image information; and a hazard prediction unit that outputs hazard information which is information about hazards predicted in the predetermined location, generated based on the detection information using basic hazard information that has been collected in advance as hazards predicted in the predetermined location.

[0013] The above object of the present invention is to enable the hazard prediction unit to output information retrieved from the basic hazard information based on the names of the detected objects included in the detection information as the hazard information, or to enable the hazard prediction unit to set an accuracy for the element information constituting the hazard information according to the number of names of the detected objects related to it, or to enable the information analysis unit to output detection information including location information indicating the location of the detected objects, and the hazard prediction unit to adjust the accuracy according to the distance between the detected objects obtained from the location information, or to enable the hazard prediction unit to use a large-scale language model learned using the basic hazard information. This can be achieved more effectively by outputting information generated based on the detection information as the hazard information; or by the information analysis unit outputting detection information including location information indicating the location of the detected object, and the hazard prediction unit outputting information generated based on the detection information using distance information between detected objects obtained from the location information as the hazard information; or by the information analysis unit outputting detection information that distinguishes between adults and children when the detected object includes both adults and children; or by the basic hazard information being composed of information collected in hazard prediction training.

[0014] Furthermore, the above objective of the present invention is more effectively achieved by a hazard prediction system comprising the above-mentioned hazard prediction device and imaging device, wherein the imaging device captures an image of the predetermined location and outputs it as image information, and the information analysis unit acquires the image information output from the imaging device, or by further comprising a terminal device for use by a user, wherein the terminal device inputs hazard information output by the hazard prediction unit, or by the information analysis unit acquiring the image information output from the imaging device via the terminal device.

[0015] Furthermore, the present invention is a hazard prediction method for predicting hazards at a predetermined location, which is performed by a computer and comprises: an information analysis step of acquiring image information of the predetermined location and acquiring or generating detection information which is information relating to detected objects detected from the image information; and a hazard prediction step of outputting hazard information which is information relating to hazards predicted at the predetermined location, generated based on the detection information using basic hazard information which has been collected in advance as hazards predicted at the predetermined location.

[0016] The above objectives of the present invention are more effectively achieved in the hazard prediction step by outputting information retrieved from the basic hazard information based on the name of the detected object included in the detection information as the hazard information, or by outputting information generated based on the detection information by a large-scale language model learned using the basic hazard information as the hazard information in the hazard prediction step.

[0017] Furthermore, the above-mentioned objectives of the present invention are achieved by a hazard prediction program that causes a computer to execute the above-mentioned hazard prediction method. [Effects of the Invention]

[0018] According to the risk prediction device, risk prediction system, risk prediction method, and risk prediction program of the present invention, the risk information is generated based on detection information using pre-collected basic risk information, so that risks can be predicted effectively and efficiently without requiring a large amount of information. [Brief Description of the Drawings]

[0019] [Figure 1] This is a diagram showing a configuration example (first embodiment) of a risk prediction system provided with a risk prediction device according to the present invention. [Figure 2] This is a sequence diagram showing an example of data transmission and reception between the risk prediction device and other devices (first embodiment). [Figure 3] This is a block diagram showing a configuration example of the risk prediction device (first embodiment). [Figure 4] This is a flowchart showing an operation example of the risk prediction device (first embodiment). [Figure 5] This is a block diagram showing a configuration example of an object detection device. [Figure 6] This is a block diagram showing a configuration example of a search device. [Figure 7] This is a block diagram showing a configuration example of a user terminal. [Figure 8] This is a diagram showing a configuration example (second embodiment) of a risk prediction system provided with a risk prediction device according to the present invention. [Figure 9] This is a block diagram showing a configuration example of the risk prediction device (second embodiment). [Figure 10] This is a diagram showing an example of fixed phrases used for prompt creation by the risk prediction unit. [Figure 11] This is a flowchart showing an operation example of the risk prediction device (second embodiment). [Figure 12] This is a block diagram showing a configuration example of a generation device. [Modes for Carrying Out the Invention]

[0020] This invention detects objects (people, things) from images (image information) taken of a designated location such as a nursery school (hereinafter referred to as the "monitoring area"), and uses information (basic hazard information) collected in advance as hazards that can be predicted in the monitoring area to predict hazards in the monitoring area based on the information (detection information) of the detected objects, and outputs information (hazard information) regarding the predicted hazards. For example, if a nursery room in a nursery school is designated as the monitoring area, children, nursery teachers, desks, toys, etc., will be detected from image information taken inside the nursery room. As basic hazard information, for example, information collected in hazard prediction training (hazard prediction exercises) can be used. Hazard prediction training is a training method to prevent accidents and disasters by predicting potential hazards in the work site or workplace in advance and considering solutions, and is also used in traffic safety education, etc. The content of potential hazards and solutions, etc., that are identified in this hazard prediction training will be used as basic hazard information. In addition, the content of past accidents, etc., that have occurred in the monitoring area may also be used as basic hazard information.

[0021] Thus, the present invention enables efficient hazard prediction with reduced costs and processing load, as it allows for hazard prediction if image information capturing at least the monitoring area is available. Furthermore, since information from effective means of hazard prediction in the monitoring area, such as hazard prediction training, can be used as basic hazard information, effective hazard prediction can be achieved.

[0022] In this invention, the name of the detected object (hereinafter referred to as "detected object name") is included in the detection information, and a process (hereinafter referred to as "search process") is performed to retrieve information from basic hazard information based on the detected object name and treat it as hazard information. For example, if the basic hazard information is a collection of sentences describing the content of the hazard, it is possible to search for sentences related to the detected object name and treat the retrieved sentences as hazard information. Furthermore, if there are multiple detected objects and the retrieved sentences are related to multiple detected object names, a probability corresponding to the number of related detected object names may be set for the retrieved sentences (element information). For example, a sentence describing a hazard with a large number of related detected object names may be considered to have a high probability of occurring, and a high probability may be set for that sentence.

[0023] In this invention, it is possible to use basic hazard information to train a large-scale language model, and then use the trained large-scale language model to generate information based on detection information, which is then treated as hazard information (hereinafter referred to as "generation process"). The large-scale language model is a language model constructed using a large amount of text data and deep learning technology. The language model is modeled using the occurrence probability of sentences and words, and is used in natural language processing such as text creation. Compared to conventional language models, the large-scale language model is a language model with a much larger computational load, data volume, and number of parameters, resulting in a language model with significantly improved accuracy. The large-scale language model is adaptable to various natural language processing tasks such as text creation, text summarization, text classification, translation, and question answering. In this invention, for example, it is possible to use basic hazard information to further train a pre-trained general-purpose large-scale language model, create input data (text, etc.) for the large-scale language model from detection information including the name of the detected object, input that input data into the further trained large-scale language model, and treat the generated text as hazard information.

[0024] In this invention, information indicating the location of detected objects (location information) is included in the image information, and the distance between detected objects determined from the location information can be used for hazard prediction. In the search process, for example, the accuracy corresponding to the number of related detected object names mentioned above may be adjusted according to the distance between detected objects. For example, if the distance between an obstacle and a person is short, the accuracy is increased, as it is assumed that there is a high probability that a hazard will occur. In the generation process, for example, information derived from the distance between detected objects may also be used to generate hazard information based on the detection information. For example, information indicating the proximity between detected objects (close, far, etc.) is generated from the distance between detected objects, and this information is also taken into account to create input data for a large-scale language model, which is then input into the large-scale language model. In this way, by using location information for hazard prediction, it becomes possible to perform more accurate hazard prediction.

[0025] In this invention, when the detected objects include both adults and children, it is possible to output detection information that distinguishes between adults and children. Since there are differences between adults and children in terms of the nature of the danger and their ability to respond to it, distinguishing between the two allows for more accurate danger prediction. In particular, in nurseries and kindergartens, the adults detected are usually childcare workers and the children detected are the children themselves, and the dangers that should be predicted accurately and without omission are those affecting the children, so it is preferable to distinguish between childcare workers (adults) and children.

[0026] The present invention may be implemented as a hazard prediction device that performs the above-described processing, or as a hazard prediction system that includes an imaging device that, in addition to the hazard prediction device, outputs image information of a monitoring area to the hazard prediction device. The hazard prediction system may also include a terminal device for use by a user, and the terminal device may input hazard information output by the hazard prediction device or input image information output from the imaging device and output it to the hazard prediction device.

[0027] In addition to the hazard prediction device and hazard prediction system, the present invention may also be implemented as a hazard prediction method that performs the above-described processing, or a hazard prediction program for executing that method.

[0028] Embodiments of the present invention will be described below with reference to the drawings. In each drawing, the same components are denoted by the same reference numerals, and their descriptions may be omitted. Furthermore, the configurations and other aspects described below are illustrative, and the present invention is not limited to these. In addition, the following description focuses on the main configurations and operations for carrying out the present invention, and general processes necessary for carrying out the present invention, such as communication processing and screen display processing, may be described in a simplified or omitted manner.

[0029] Figure 1 shows an example configuration of a hazard prediction system (first embodiment) equipped with a hazard prediction device according to the present invention. The hazard prediction system 1 of this embodiment is a system that predicts hazards in a nursery school, with the nursery school as the monitoring area. It detects objects from image information captured from the inside of the nursery room, etc., and predicts hazards through search processing.

[0030] The hazard prediction system 1 comprises an imaging device 2 that images a monitoring area such as a nursery room, an object detection device 4 that performs object detection processing from image information (hereinafter referred to as "object detection processing"), a search device 5 that performs search processing, a hazard prediction device 3 that performs hazard prediction in cooperation with the object detection device 4 and the search device 5, and a user terminal (terminal device) 6 used by the user, all of which are connected via a network 10 consisting of the internet or the like. For example, a general-purpose server is used as the object detection device 4, the search device 5, and the hazard prediction device 3, and the processing necessary for hazard prediction is executed by a program running on the server. As the user terminal 6, a general-purpose information device with communication capabilities such as a personal computer or tablet is used, and processing on the user terminal 6 can also be executed by a program (application program). In the hazard prediction system 1, the object detection processing and search processing are performed by a cloud service (cloud computing service) available through the network 10. Therefore, the servers used by the service provider that provides these cloud services become the object detection device 4 and the search device 5. Regarding the hazard prediction device 3, it is also possible to provide hazard prediction as a cloud service by using a server for cloud services as the device. Furthermore, if the cloud service that performs object detection processing and the cloud service that performs search processing are provided by the same service provider, the object detection device 4 and the search device 5 may be on the same server. Also, at least one of the hazard prediction device 3, object detection device 4, search device 5, and user terminal 6 may be a dedicated device rather than a general-purpose device, and some or all of the processing may be implemented in hardware.

[0031] In the hazard prediction system 1, the hazard prediction device 3 acts as the central hub, transmitting and receiving data via the network 10 between the imaging device 2, object detection device 4, search device 5, and user terminal 6 to predict hazards in the nursery school. The flow of data transmission and reception between the hazard prediction device 3 and the imaging device 2, object detection device 4, search device 5, and user terminal 6 will be explained in outline. Note that while data transmission and reception between devices may involve processes such as sending the data reception result (normal, abnormal, etc.) back to the sending device, such general processes will be omitted from this explanation.

[0032] Figure 2 shows an example of data transmission and reception between the hazard prediction device 3 and other devices. Figure 2 is a sequence diagram illustrating an example of data transmission and reception. Note that the flow of data transmission and reception between the hazard prediction device 3 and other devices is not limited to the example below.

[0033] The user terminal 6 transmits a start signal to the hazard prediction device 3 in order to predict hazards in the nursery school (step S1). However, if hazard prediction is performed continuously, or if the user terminal 6 does not need to transmit a start signal to the hazard prediction device 3, for example, if the user terminal 6 is instructed to start by using the input means provided by the hazard prediction device 3.

[0034] Upon receiving the start signal, the hazard prediction device 3 transmits a signal to the imaging device 2 requesting imaging (hereinafter referred to as the "imaging request signal") (step S2). When the imaging device 2 receives the imaging request signal, it images the monitoring area, such as the nursery room, and transmits the image information to the hazard prediction device 3.

[0035] When the hazard prediction device 3 receives image information, it transmits it to the object detection device 4 (step S3). The object detection device 4 performs object detection processing on the received image information and transmits the detection result to the hazard prediction device 3 as detection information.

[0036] When the hazard prediction device 3 receives detection information, it transmits the information necessary for the search process, such as the name of the detected object, contained in the detection information, to the search device 5 as search information (step S4). The search device 5 performs a search process based on the received search information and transmits the retrieved information to the hazard prediction device 3 as search hazard information.

[0037] When the hazard prediction device 3 receives search hazard information, it transmits hazard information generated based on that information to the user terminal 6. The user terminal 6 displays the received hazard information using a display means such as a display or liquid crystal panel provided in the user terminal 6.

[0038] The imaging device 2, hazard prediction device 3, object detection device 4, search device 5, and user terminal 6 that constitute the hazard prediction system 1 described above will be explained below.

[0039] First, let me explain the imaging device 2. The imaging device 2 is installed within the nursery school where hazard prediction is performed. It captures images of the monitoring area and transmits the captured image information to the hazard prediction device 3. Multiple imaging devices 2 can be installed. Since children may take unexpected actions and hazards lurk in various places within the nursery school, the imaging devices 2 should be installed in appropriate locations, taking into consideration privacy concerns. For example, imaging devices 2 can be installed in classrooms, entrances, corridors, and playgrounds.

[0040] For example, a camera with communication capabilities can be used as the imaging device 2. The image information transmitted by the imaging device 2 to the hazard prediction device 3 can be either still images or videos. In other words, if the image information that can be targeted by the object detection processing by the object detection device 4 can be either still images or videos, the hazard prediction device 3 should transmit image information to the object detection device 4 that matches the form of the image information transmitted from the imaging device 2 (still images, videos). If the image information that the object detection device 4 can target is only still images, and the imaging device 2 transmits video image information, the hazard prediction device 3 should extract still images from the received image information and transmit them to the object detection device 4.

[0041] When the imaging device 2 receives an imaging request signal transmitted from the hazard prediction device 3, it images the monitoring area, such as a nursery room. The image of the monitoring area captured is transmitted to the hazard prediction device 3 as image information. Alternatively, the imaging device 2 may automatically capture images of the monitoring area at regular time intervals and transmit the captured image information to the hazard prediction device 3. In this case, it is not necessary for the hazard prediction device 3 to transmit an imaging request signal. Furthermore, the imaging device 2 may be a dedicated device for hazard prediction, or it may be a device used for other purposes, such as a surveillance camera. In the latter case, for example, the hazard prediction device 3 may receive image information from a device that controls the imaging device 2.

[0042] Next, I will explain the hazard prediction device 3. The hazard prediction device 3 uses image information transmitted from the imaging device 2 to perform hazard prediction in the monitoring area captured in the image information, in cooperation with the object detection device 4 and the search device 5. Figure 3 is a block diagram showing an example configuration of the hazard prediction device 3. A general-purpose server is used as the hazard prediction device 3, and Figure 3 shows only the components related to the present invention. This is also true for the example configurations of the object detection device 4, the search device 5, and the user terminal 6 described later, as well as for the diagrams showing example configurations of each device in other embodiments.

[0043] The hazard prediction device 3 includes an information analysis unit 31 that cooperates with an object detection device 4 that performs object detection processing, and a hazard prediction unit 32 that cooperates with a search device 5 that performs search processing.

[0044] The start signal Stt transmitted from the user terminal 6 is input to the information analysis unit 31. Upon receiving the start signal Stt, the information analysis unit 31 outputs an imaging request signal Rqt requesting imaging of the monitoring area by the imaging device 2. The imaging request signal Rqt is transmitted to the imaging device 2. Upon receiving the imaging request signal Rqt, the imaging device 2 images the monitoring area and transmits the image information Img to the hazard prediction device 3.

[0045] The information analysis unit 31 receives image information Img transmitted from the imaging device 2, outputs image information ImgA to cause the object detection device 4 to perform object detection processing, and transmits image information ImgA to the object detection device 4. If image information Img can be used directly for object detection processing by the object detection device 4, the information analysis unit 31 transmits image information Img as image information ImgA. If image information Img cannot be used directly for object detection processing by the object detection device 4 due to differences in format, etc., the information analysis unit 31 modifies the format of image information Img and transmits it as image information ImgA. Alternatively, the information analysis unit 31 may perform preprocessing on image information Img, such as image centering or trimming to remove unnecessary parts, using known methods, and then transmit it as image information ImgA.

[0046] The object detection device 4, upon receiving the image information ImgA, performs object detection processing on the image information ImgA and transmits the detection result as detection information Dct to the hazard prediction device 3. The detection information Dct includes the names of the detected objects (detected object names) found in the image information ImgA. For example, if a nursery room is designated as the monitoring area, nursery teachers, children, toys, desks, chairs, etc., will be detected from the image information ImgA, and their names will be output as detected object names. Details of the object detection processing by the object detection device 4 will be described later.

[0047] The information analysis unit 31 receives detection information Dct transmitted from the object detection device 4 and outputs detection information DctA. The information analysis unit 31 may output detection information Dct as detection information DctA, or it may extract only the necessary information from detection information Dct and output it as detection information DctA. Detection information DctA is input to the hazard prediction unit 32.

[0048] The hazard prediction unit 32 receives detection information DctA and outputs search information Srh to be sent to the search device 5 in order to execute the search process. Normally, since detection information DctA contains the names of multiple detected objects, the hazard prediction unit 32 sequentially outputs each detected object name in the search information Srh. In this case, if it is necessary to change the data format or other aspects in order to make it a target for search processing by the search device 5, the unit performs that processing and outputs it as search information Srh.

[0049] Upon receiving the search information Srh transmitted from the hazard prediction device 3, the search device 5 performs a search based on the search information Srh and transmits the search results to the hazard prediction device 3 as search hazard information Dsr. Based on the name of the object being searched for included in the search information Srh, the search device 5 searches for the basic hazard information collected in advance as hazards predicted in the monitoring area and transmits the results as search hazard information Dsr. As basic hazard information, the search device 5 uses, for example, documents describing the content of hazards collected in hazard prediction training (hereinafter referred to as "predicted hazard documents"), and includes all predicted hazard documents searched based on the name of the object being searched in the search hazard information Dsr. Each retrieved predicted hazard document corresponds to element information. In addition, each predicted hazard document is uniquely assigned an identification number (ID) (hereinafter referred to as "document ID") to distinguish it from other predicted hazard documents, and the search hazard information Dsr also includes the document ID assigned to the retrieved predicted hazard document. Details of the search process by the search device 5 will be described later.

[0050] The hazard prediction unit 32 receives the search hazard information Dsr as input, and once it has received the search hazard information Dsr for all detected object names included in the detection information DctA, it generates and outputs hazard information Dgr based on that information. Specifically, the hazard prediction unit 32 includes the predicted hazard sentences included in the search hazard information Dsr and the probability of each predicted hazard sentence occurring (hereinafter referred to as "hazard occurrence rate") in the hazard information Dgr and outputs it. The hazard information Dgr is transmitted to the user terminal 6.

[0051] The hazard prediction unit 32 determines the degree of hazard occurrence according to the number of detected object names associated with the predicted hazard text. In other words, the hazard prediction unit 32 increases the degree of hazard occurrence for predicted hazard texts that have a large number of associated detected object names, assuming that the content of the predicted hazard text is likely to occur. Specifically, the hazard prediction unit 32 counts the number of times each predicted hazard text is included in all the input search hazard information Dsr, and determines the degree of hazard occurrence for each predicted hazard text based on this count. Since each predicted hazard text is assigned a text ID, the hazard prediction unit 32 uses the text ID as an index to count the number of times each predicted hazard text is included. In other words, when search hazard information Dsr is input, the hazard prediction unit 32 increments the count for the predicted hazard texts included in that search hazard information Dsr by one. This process is continued until search hazard information Dsr for all detected object names included in detection information DctA is input, and the degree of hazard occurrence is determined based on the count for each predicted hazard text. For example, the hazard prediction unit 32 sets four levels of hazard occurrence, A, B, C, and D, in descending order of hazard occurrence, and sets a threshold for the count for each level to determine which level each predicted hazard document belongs to. For example, counts of 1, 2, 3, and 4 or more are designated as hazard occurrence levels D, C, B, and A, respectively. Alternatively, the predicted hazard document itself may be used as an index instead of the document ID. In this case, the document ID is not required. The hazard prediction unit 32 may also output hazard information Dgr without including predicted hazard documents with low hazard occurrence levels in the hazard information Dgr. The hazard prediction unit 32 may also store the predicted hazard documents in order of hazard occurrence level and output hazard information Dgr. In this case, the hazard prediction unit 32 does not need to include the hazard occurrence level in the hazard information Dgr.

[0052] In this configuration of the hazard prediction device 3, an example of its operation will be explained with reference to the flowchart in Figure 4.

[0053] When the information analysis unit 31 receives a start signal Stt from the user terminal 6 (step S10), it outputs an imaging request signal Rqt, which is then transmitted to the imaging device 2 (step S20). The imaging device 2 receives the imaging request signal Rqt and transmits the image information Img of the monitoring area to the hazard prediction device 3. When the information analysis unit 31 receives the image information Img (step S30), it outputs image information ImgA, which is then transmitted to the object detection device 4 (step S40).

[0054] The object detection device 4, upon receiving the image information ImgA, performs object detection processing and transmits the detection information Dct to the hazard prediction device 3. When the information analysis unit 31 receives the detection information Dct transmitted to the hazard prediction device 3 (step S50), it outputs the detection information DctA to the hazard prediction unit 32.

[0055] The hazard prediction unit 32 sequentially reads out the names of the detected objects contained in the input detection information DctA (step S60), outputs them as search information Srh, and transmits the search information Srh to the search device 5 (step S70).

[0056] The search device 5, having received the search information Srh, performs the search process and transmits the search hazard information Dsr to the hazard prediction device 3. When the hazard prediction unit 32 receives the search hazard information Dsr transmitted to the hazard prediction device 3 (step S80), it reads and stores all the predictive hazard sentences contained in the search hazard information Dsr (step S90) and increments the count of each predictive hazard sentence by one (step S100). Note that if the read predictive hazard sentence is already stored, the hazard prediction unit 32 does not need to store the predictive hazard sentence again.

[0057] When the hazard prediction unit 32 reads all the names of detected objects included in the detection information DctA (step S110), it determines the degree of hazard occurrence for each of the stored predicted hazard sentences based on the count (step S120). Then, the hazard prediction unit 32 outputs hazard information Dgr, which includes the predicted hazard sentences and the degree of hazard occurrence, and transmits the hazard information Dgr to the user terminal 6 (step S130).

[0058] I will now describe the object detection device 4. The object detection device 4 performs object detection processing on the image information ImgA transmitted from the hazard prediction device 3. Since the object detection processing in the hazard prediction system 1 is performed by a cloud service available through the network 10, the object detection device 4 is a server used by the service provider that provides that cloud service.

[0059] Figure 5 shows an example of the configuration of the object detection device 4. The object detection device 4 comprises a control unit 41, which consists of a CPU (Central Processing Unit) and the like, that is equipped in a general-purpose server, and a storage unit 42, which consists of memory and storage. Image information ImgA transmitted from the hazard prediction device 3 is input to the control unit 41, and detection information Dct, which is the detection result in the object detection process, is output from the control unit 41 and transmitted to the hazard prediction device 3.

[0060] The object detection device 4 performs object detection using an object detection algorithm based on machine learning, such as deep learning or convolutional neural networks. Such object detection algorithms require training on images of the objects to be detected, but object detection algorithms provided as cloud services usually have a pre-trained model (hereinafter referred to as the "pre-trained detection model"). When detecting only objects that can be handled by the pre-trained detection model, the object detection process is performed using the pre-trained detection model. However, the hazard prediction system 1 uses a nursery school as its monitoring area, and needs to perform object detection specific to the monitoring area, such as distinguishing between childcare workers and children from the image information ImgA of the monitoring area. Therefore, the object detection device 4 performs additional training based on the pre-trained detection model and performs object detection using the model created through additional training (hereinafter referred to as the "additional detection training model"). For example, in order to detect childcare workers, the hazard prediction system 1 prepares image data of childcare workers wearing uniforms, and uses this image data to perform additional training with the object detection device 4. Since many daycare centers standardize the clothing worn by childcare workers, such as aprons, it is possible to distinguish and detect childcare workers by performing additional training using image data captured while wearing these uniforms. In this case, the accuracy of childcare worker detection can be improved by preparing a separate detection additional training model for each daycare center. If it is necessary to detect objects other than childcare workers that cannot be handled by the pre-trained detection model, image data of those objects can be prepared and additional training can be performed using the object detection device 4. Furthermore, since childcare workers are adults and children are children, the object detection device 4 may also undergo additional training to distinguish between adults and children.

[0061] Image data used for additional training is transmitted, for example, from the hazard prediction device 3 to the object detection device 4, and the object detection device 4 performs additional training based on instructions from the hazard prediction device 3. The detection additional training model created during the additional training is stored in the memory unit 42 of the object detection device 4. The detection pre-training model is also stored in the memory unit 42.

[0062] The object detection device 4 uses the detection additional learning model (or the detection pre-learning model if additional learning is not required) stored in the memory unit 42 to perform object detection processing on the image information ImgA via the control unit 41. The object detection algorithm performs a detection process to find the region where objects exist and an identification process to determine the objects present within the found region. Therefore, the control unit 41 performs the detection and identification processes on the image information ImgA to detect objects such as childcare workers, children, and toys. The control unit 41 then outputs the names of all detected objects (detected object names) as detection information Dct, and the detection information Dct is transmitted to the hazard prediction device 3. There are various object detection algorithms, including algorithms that perform the detection and identification processes consecutively and algorithms that perform both processes simultaneously, but the control unit 41 may use any algorithm.

[0063] An example of the operation of the object detection process in the object detection device 4 with this configuration is as described above.

[0064] I will now describe the search device 5. The search device 5 performs search processing based on the search information Srh transmitted from the hazard prediction device 3. The search processing in the hazard prediction system 1, like the object detection processing, is performed by a cloud service available through the network 10, so the search device 5 is a server used by the service provider that provides that cloud service.

[0065] Figure 6 shows an example of the configuration of the search device 5. Similar to the object detection device 4, the search device 5 comprises a control unit 51 consisting of a CPU (Central Processing Unit) etc., which is equipped in a general-purpose server, and a storage unit 52 consisting of memory, storage etc. The search information Srh transmitted from the hazard prediction device 3 is input to the control unit 51, and the search hazard information Dsr, which is the search result of the search process, is output from the control unit 51 and transmitted to the hazard prediction device 3.

[0066] The search device 5 performs a search by searching against basic hazard information collected in advance as foreseeable hazards in the monitoring area, and uses foreseeable hazard texts collected in hazard prediction training as basic hazard information. Hazard prediction training is a training method to prevent accidents and disasters by predicting potential hazards in the monitoring area in advance and considering solutions. Hazard prediction training is usually carried out in four steps: "understanding the current situation," "pursuing the essence," "establishing countermeasures," and "setting goals," using illustrations or images (hereinafter referred to as "training samples") that show the situation in the monitoring area. In "understanding the current situation," potential hazards in the situation shown in the training samples are extracted; in "pursuing the essence," important hazards are extracted from the hazards extracted in "understanding the current situation"; in "establishing countermeasures," countermeasures are made for the important hazards extracted in "pursuing the essence"; and in "setting goals," important countermeasures are extracted from the countermeasures made in "establishing countermeasures," and final action goals are set. In the hazard prediction system 1, for example, images of a nursery room or other area that is a monitoring area are used as training samples, and hazard prediction training is conducted with childcare workers as participants. The content of the hazards extracted through "understanding the current situation" and / or "pursuing the essence" in the hazard prediction training is used as predicted hazard text. For example, if a nursery room is used as a monitoring area, predicted hazard texts may include "a desk might fall over and hit a child," "something placed on a shelf might fall," and "a child might trip over a toy." Note that the search device 5 may not use the content of the hazards extracted in the hazard prediction training as predicted hazard text, but may also use texts that have been adjusted, such as by combining similar hazards into one.

[0067] The search device 5 performs the search process for the basic hazard information described above using vector search. Vector search is a search method that finds related information by converting data such as text and images into numerical vectors and calculating the distance (similarity) between those vectors. Because it is possible to search while considering the meaning and context of the data, it is said that vector search can find more relevant information. A deep learning model is used to convert the data into vectors in vector search. The deep learning model used for this vector conversion is stored in the memory unit 52. In addition, the predictive hazard documents that are the target of the search also need to be converted into vectors, so in the hazard prediction system 1, for example, predictive hazard documents are sent from the hazard prediction device 3 to the search device 5, and the predictive hazard documents converted into vectors are stored in the memory unit 52 as basic hazard information. Since each predictive hazard document is assigned a document ID, the document ID for each predictive hazard document is also stored in the memory unit 52.

[0068] The control unit 51 converts the detected object names contained in the input search information Srh into vectors, calculates the similarity with the basic hazard information stored in the memory unit 52, and searches for predictive hazard texts related to the detected object names. The control unit 51 outputs the retrieved predictive hazard texts along with the text IDs in the search hazard information Dsr, and the search hazard information Dsr is transmitted to the hazard prediction device 3.

[0069] An example of the operation of the search process in the search device 5 with this configuration is as described above.

[0070] Although the search device 5 performs the search using vector search, it may also perform the search using other search methods, such as full-text search or semantic search.

[0071] Finally, I will explain user terminal 6. User terminal 6 is a device used by users of the hazard prediction system 1, and a general-purpose information device is used as user terminal 6.

[0072] Figure 7 shows an example of the configuration of the user terminal 6. The user terminal 6 comprises an input means 62 consisting of a keyboard, mouse, touch panel, etc., which are provided by general-purpose information equipment; a display means 63 consisting of a display, liquid crystal panel, etc., which are provided by general-purpose information equipment; and a control unit 61 that controls the information transmitted and received between the user terminal 6 and the hazard prediction device 3.

[0073] The control unit 61 outputs a start signal Stt to initiate hazard prediction in the monitoring area, and the start signal Stt is transmitted to the hazard prediction device 3. For example, the control unit 61 displays a button for initiating hazard prediction on the screen using the display means 63, and outputs the start signal Stt when the button is touched or clicked by the input means 62.

[0074] The hazard information Dgr transmitted from the hazard prediction device 3 is input to the control unit 61, and the control unit 61 displays the predicted hazard text and the degree of hazard occurrence included in the input hazard information Dgr using the display means 63. For example, the control unit 61 displays the predicted hazard text and the degree of hazard occurrence in pairs on the screen in bullet points, in descending order of hazard occurrence. Note that the degree of hazard occurrence does not have to be displayed. The user terminal 6 may also output the predicted hazard text as audio.

[0075] An example of the operation of user terminal 6 with this configuration is as described above.

[0076] The following modifications are possible to this embodiment. In the hazard prediction system 1, the image information Img transmitted from the imaging device 2 is received by the hazard prediction device 3, but it may also be received by the hazard prediction device 3 via the user terminal 6. That is, the user terminal 6 receives the image information Img transmitted from the imaging device 2 and transmits the image information Img from the user terminal 6 to the hazard prediction device 3. In this case, the imaging request signal Rqt transmitted from the hazard prediction device 3 may be transmitted to the imaging device 2 via the user terminal 6, or the user terminal 6 may transmit the imaging request signal Rqt independently without receiving the imaging request signal Rqt from the hazard prediction device 3. This allows the user terminal 6 to retain image information, and for example, when the hazard prediction system 1 described later is used for education, past image information can also be used, enabling efficient education in various cases.

[0077] The object detection device 4 outputs the name of the detected object as detection information Dct. However, some object detection algorithms also output information indicating the location of the detected object in image information ImgA (location information), information indicating the certainty of the detected object (hereinafter referred to as "confidence level"), etc., in addition to the name of the detected object. Therefore, this information may also be included in the detection information Dct and transmitted to the hazard prediction device 3. For location information, the object detection device 4 may, for example, surround the location where the detected object exists in image information ImgA with a frame (rectangle), and use the coordinates of the center of the frame and the vertical and horizontal lengths of the frame. The confidence level is the probability that the detected object exists within that frame. In the hazard prediction device 3, the information analysis unit 31 outputs the location information and / or confidence level included in detection information DctA to the hazard prediction unit 32, and the hazard prediction unit 32 may use the location information and / or confidence level to generate hazard information Dgr.

[0078] For example, if the detected object name includes "child," the hazard prediction unit 32 uses location information to calculate the distance between the child and other detected objects, and determines whether the detected object is close to or far from the child by setting a threshold (hereinafter referred to as "proximity determination"). The hazard prediction unit 32 then uses the result of the proximity determination to determine the degree of hazard occurrence. For example, the hazard prediction unit 32 may adjust the degree of hazard occurrence for a predicted hazard statement by increasing it if the detected object corresponding to the detected object name included in the predicted hazard statement is close to the child, or by decreasing it if the detected object is far from the child.

[0079] Regarding reliability, for example, the hazard prediction unit 32 may set a threshold for reliability, and if it is unclear whether a detected object exists whose reliability is below the threshold, it may choose not to transmit the name of the detected object as search information Srh to the search device 5.

[0080] When the object detection device 4 transmits detection information Dct, which includes the name of the detected object, location information, and confidence level, to the hazard prediction device 3, the hazard prediction device 3 may also transmit the detected object name, location information, confidence level, and image information ImgA to the user terminal 6, along with the predicted hazard text and hazard occurrence level. The user terminal 6 displays the predicted hazard text and hazard occurrence level on the screen using the display means 63, and also displays the image information ImgA on the screen, overlaying the frame indicated by the location information, the detected object name, and the confidence level onto the image. This allows the user to confirm the results of the object detection process. Furthermore, the image information displayed by the user terminal 6 may be the image information Img input to the information analysis unit 31 or the image information ImgA output from the information analysis unit 31, or the image information that the object detection device 4 may include in the detection information Dct and transmit, and that image information may be used, or if the image information Img transmitted from the imaging device 2 is received by the hazard prediction device 3 via the user terminal 6, the user terminal 6 may hold and use that image information Img.

[0081] If the hazard prediction device 3 can determine the speed and direction of movement of a detected object, for example, by having detection information Dct from the object detection device 4 containing positional information and image information Img from the imaging device 2 being a video, the hazard prediction unit 32 may generate hazard information Dgr using the speed and direction of movement of the detected object. For example, the hazard prediction unit 32 may determine whether the detected object is approaching or moving away (hereinafter referred to as "movement determination"), and adjust the hazard level of the predicted hazard text if the detected object corresponding to the name of the detected object included in the predicted hazard text is approaching the child, or lower it if the detected object is moving away from the child.

[0082] The hazard prediction system 1 may also use information that identifies the monitoring area where the image information Img was captured (hereinafter referred to as "monitoring area information") in its search process. For example, training samples used in hazard prediction training are prepared for each monitoring area, such as a nursery room or playground, and the predicted hazard sentences collected in the hazard prediction training using each training sample are classified by monitoring area. The search device 5 then stores the predicted hazard sentences for each monitoring area as basic hazard information in the storage unit 52. The hazard prediction device 3 transmits the monitoring area information along with the search information Srh to the search device 5, and the search device 5 performs the search process using the basic hazard information corresponding to the received monitoring area information. This enables more accurate search processing. The hazard prediction device 3, for example, has in advance information that associates information identifying the imaging device 2 that transmits the image information Img (e.g., an IP address) with the monitoring area where the imaging device 2 is installed, identifies the imaging device 2 that transmitted the received image information Img, and obtains the monitoring area information by using the associated information. If image information (Img) is received by the hazard prediction device 3 via the user terminal 6, the user terminal 6 may also transmit the monitoring area information to the hazard prediction device 3. The hazard prediction system 1 may also utilize the monitoring area information in object detection processing, for example, by preparing an additional detection learning model for each monitoring area used by the object detection device 4 in object detection processing.

[0083] The hazard prediction system 1 may also use information that identifies the scene in which the image information Img was captured (hereinafter referred to as the "imaged scene") (hereinafter referred to as "scene information") for the search process. For example, the imaged scenes may include scenes of structured childcare (group childcare) conducted based on a lesson plan set by a childcare worker, mealtime scenes, naptime scenes, etc., and images are taken for each imaged scene. Then, for example, similar to the case where monitoring area information is used, the predicted hazard sentences collected in the hazard prediction training are classified by imaged scene, and the search device 5 stores the predicted hazard sentences for each imaged scene as basic hazard information in the storage unit 52. The hazard prediction device 3 transmits the scene information along with the search information Srh to the search device 5, and the search device 5 performs the search process using the basic hazard information corresponding to the received scene information. This allows for more accurate search processing, similar to the case where monitoring area information is used. The hazard prediction device 3 determines the imaged scene based on, for example, the time the monitoring area was imaged, and acquires the scene information. Since the times for scheduled childcare, meals, naps, etc., are generally fixed, if the scenes in which these activities take place are to be used as imaging scenes, the imaging scenes can be determined based on the time. The hazard prediction device 3 may also acquire the imaging scene estimated from the image information Img as scene information using a model learned using the image information collected for each imaging scene. Alternatively, a user may look at the image captured by the imaging device 2 to determine the imaging scene, input the determined imaging scene using the input means 62 of the user terminal 6, and the user terminal 6 transmits this information as scene information to the hazard prediction device 3, thereby acquiring the scene information.

[0084] The search device 5 includes the similarity between the retrieved predicted hazardous text and the searched object name in the search hazard information Dsr and transmits it to the hazard prediction device 3. The hazard prediction device 3 may use the similarity to determine the degree of hazard occurrence, etc. For example, the hazard prediction unit 32 of the hazard prediction device 3 may make adjustments such as lowering the degree of hazard occurrence for predicted hazardous texts whose similarity is less than a predetermined threshold. The hazard prediction device 3 may also transmit hazard information Dgr, which does not include predicted hazardous texts with low similarity, to the user terminal 6.

[0085] The hazard prediction device 3 performs object detection processing and search processing in cooperation with the object detection device 4 and the search device 5, respectively, but either or both of these processes may be performed by the hazard prediction device 3 alone.

[0086] Other embodiments of the present invention will be described. The hazard prediction system 1 of the first embodiment predicts hazards through a search process, but it is also possible to predict hazards through a generation process using a large-scale language model.

[0087] Figure 8 shows an example configuration of a hazard prediction system that predicts dangers by performing a generation process (second embodiment). The hazard prediction system 1A of the second embodiment, like the hazard prediction system 1 of the first embodiment, is a system that predicts dangers in a nursery school, with the nursery school as the monitoring area. It detects objects from image information captured from the inside of the nursery rooms, etc., and predicts dangers by performing a generation process rather than a search process. Compared to the hazard prediction system 1 shown in Figure 1, the hazard prediction device 3 in the hazard prediction system 1A is replaced by the hazard prediction device 3A, and the search device 5 is replaced by a generation device 7 that performs generation processing.

[0088] The hazard prediction device 3A uses image information transmitted from the imaging device 2 to perform hazard prediction in the monitoring area captured in the image information, in cooperation with the object detection device 4 and the generation device 7. Figure 9 is a block diagram showing an example configuration of the hazard prediction device 3A. A general-purpose server is used as the hazard prediction device 3A, and the hazard prediction device 3A performs the necessary processing for hazard prediction by a program running on the server.

[0089] The hazard prediction device 3A has the same configuration as the hazard prediction device 3 shown in Figure 3, and includes an information analysis unit 31A that cooperates with the object detection device 4 which performs object detection processing, and a hazard prediction unit 32A that cooperates with the generation device 7 which performs generation processing.

[0090] The information analysis unit 31A performs the same processing as the information analysis unit 31, but the object detection device 7 of the hazard prediction system 1A performs object detection processing on the image information ImgA and transmits detection information DctB, which includes the name of the detected object as well as location information indicating the position of the detected object in the image information ImgA, to the hazard prediction device 3A. The information analysis unit 31A receives the detection information DctB as input and outputs detection information DctC, which includes location information.

[0091] The hazard prediction unit 32A receives detection information DctC and outputs information Gnt necessary for the generation device 7 to perform the generation process (hereinafter referred to as "generation information"). The generation device 7 performs the generation process using a large-scale language model learned using basic hazard information. When using a large-scale language model, appropriate response sentences are usually generated by inputting command sentences or question sentences called prompts. Therefore, the hazard prediction unit 32A creates a prompt using the detection information DctC, outputs it as generation information Gnt, and transmits the generation information Gnt to the generation device 7.

[0092] To create a prompt, the hazard prediction unit 32A first uses the location information contained in the detection information DctC to determine whether the detected object is close to or far from the child. Similar to the modification of the first embodiment, the hazard prediction unit 32A calculates the distance between the child and other detected objects from the child's location information (e.g., the coordinates of the center of the frame) and the location information of other detected objects. If the distance is shorter than a predetermined threshold, it determines that the detected object is close to the child; if it is greater than or equal to the threshold, it determines that the object is far from the child. Based on this determination result and the name of the detected object contained in the detection information DctC, the hazard prediction unit 32A creates a prompt. Note that the distance between detected objects may also be determined and used to create the prompt, not just the distance from the child (far / close).

[0093] The hazard prediction unit 32A prepares standard phrases in advance, for example, as shown in Figure 10. In the standard phrases shown in Figure 10, the name of the detected object is inserted in [Detected Object Name], and the result of the distance determination ("near" or "far") is inserted in [Distance Determination Result]. The hazard prediction unit 32A creates a prompt by applying the name of the detected object and the distance determination result contained in the detection information DctC to the standard phrase, and outputs it as generation information Gnt. Alternatively, the display means provided by the hazard prediction device 3A may display the name of the detected object and the distance determination result, and the administrator of the hazard prediction system 1A may input the prompt using an input means such as a keyboard provided by the hazard prediction device 3A, and the hazard prediction unit 32A may output that prompt as generation information Gnt.

[0094] Upon receiving the generation information Gnt, the generation device 7, as described above, performs generation processing based on the generation information Gnt using a large-scale language model trained with basic hazard information, and transmits the generated response sentences, etc. (hereinafter referred to as "response hazard sentences") to the hazard prediction device 3A as generated hazard information Dgt. As basic hazard information, the generation device 7 uses predictive hazard sentences collected in hazard prediction training, for example, as in the case of the search device 5. Details of the generation processing by the generation device 7 will be described later.

[0095] The hazard prediction unit 32A receives generated hazard information Dgt as input and outputs hazard information Dgr. The hazard prediction unit 32A may output generated hazard information Dgt as hazard information Dgr, or it may modify the text contained in generated hazard information Dgt and output it as hazard information Dgr. The hazard information Dgr is transmitted to the user terminal 6.

[0096] In this configuration of the hazard prediction device 3A, an example of its operation will be explained with reference to the flowchart in Figure 11.

[0097] The hazard prediction device 3A operates in the same manner as the hazard prediction device 3 until the information analysis unit 31A receives the detection information DctB transmitted from the object detection device 7 and outputs the detection information DctC to the hazard prediction unit 32A (steps S10 to S50).

[0098] The hazard prediction unit 32A performs distance determination using the location information contained in the input detection information Dct (step S210). Then, the hazard prediction unit 32A creates a prompt from the result of the distance determination and the name of the detected object contained in the detection information DctC, outputs it as generation information Gnt, and transmits the generation information Gnt to the generation device 7 (step S220).

[0099] The generation device 7, having received the generation information Gnt, performs the generation process and transmits the generated hazard information Dgt to the hazard prediction device 3A. When the hazard prediction unit 32A receives the generated hazard information Dgt transmitted to the hazard prediction device 3A (step S230), it outputs hazard information Dgr, which is then transmitted to the user terminal 6 (step S240).

[0100] The generating device 7 will now be described. The generation device 7 performs generation processing based on the generation information Gnt transmitted from the hazard prediction device 3A. Similar to the search device 5, the generation device 7 uses a general-purpose server and performs generation processing using a program running on the server. Furthermore, the generation processing in the hazard prediction system 1A is performed via a cloud service available through the network 10, similar to the search processing in the hazard prediction system 1. Therefore, the generation device 7 is a server used by the service provider that provides that cloud service. Note that if the cloud service that performs the generation processing is provided by the same service provider that provides the cloud service that performs the object detection processing, the generation device 7 may be the same server as the object detection device 4. Also, the generation device 7 may be a dedicated device rather than a general-purpose device, and some or all of the processing may be implemented in hardware.

[0101] Figure 12 shows an example of the configuration of the generation device 7. Similar to the search device 5, the generation device 7 comprises a control unit 71 consisting of a CPU (Central Processing Unit) etc., which is equipped in a general-purpose server, and a storage unit 72 consisting of memory, storage etc. The generation information Gnt transmitted from the hazard prediction device 3A is input to the control unit 71, and the generated hazard information Dgt, which includes the response hazard text, is output from the control unit 71 and transmitted to the hazard prediction device 3A.

[0102] The generation device 7 performs generation processing using a large-scale language model. A large-scale language model is a language model constructed using a large amount of text data and deep learning technology. Large-scale language models used in cloud services are usually models that have been pre-trained using a large amount of text data (hereinafter referred to as the "generation pre-trained model"). The generation device 7 can also perform generation processing using the generation pre-trained model, but in order to generate more appropriate response danger sentences, it performs additional training based on the generation pre-trained model using basic danger information, and performs generation processing using the model created through additional training (hereinafter referred to as the "generation additional training model"). As basic danger information, the generation device 7 uses the predictive danger sentences collected in danger prediction training, which are also used in the search device 5.

[0103] Basic hazard information used for additional learning is transmitted, for example, from the hazard prediction device 3A to the generation device 7. The generation device 7 then performs additional learning according to instructions from the hazard prediction device 3A and stores the generated additional learning model created during the additional learning in the storage unit 72. The generated pre-learning model is also stored in the storage unit 72.

[0104] The control unit 71 generates a dangerous response sentence for the prompt contained in the input generation information Gnt using the generation and additional learning model stored in the memory unit 72, and outputs it as generation dangerous information Dgt.

[0105] An example of the operation of the generation process in the generation device 7 with this configuration is as described above.

[0106] Furthermore, the hazard prediction unit 32A of the hazard prediction device 3A does not need to use location information to create prompts when it is necessary to simplify the creation of prompts. In this case, the hazard prediction unit 32A does not need to perform distance determination. Conversely, when it is necessary to create more specific prompts, the hazard prediction unit 32A may use monitoring area information and / or scene information to create prompts, as in the modified example of the first embodiment. When using monitoring area information, for example, the hazard prediction unit 32A prepares a standard phrase such as, "Please tell me about the dangers that occur in the following situations in the [monitoring area information] of the nursery school," and creates a prompt by inserting the monitoring area information into [monitoring area information]. When using scene information, for example, the hazard prediction unit 32A prepares a standard phrase such as, "Please tell me about the dangers that occur in the following situations during the [scene information] time at the nursery school," and creates a prompt by inserting the scene information into [scene information]. By using monitoring area information and / or scene information, the hazard prediction unit 32A may be able to obtain more appropriate response danger texts. The hazard prediction device 3A can acquire monitoring area information and scene information by processing similarly to that of the hazard prediction device 3, for example.

[0107] The hazard prediction device 3A may, as in the modified version of the first embodiment, perform a movement determination if it can determine the movement speed and direction of the detected object, and use the determination result to create a prompt. For example, the hazard prediction unit 32A may prepare a standard phrase such as "A child is [moving towards] [object name]," and create a prompt by inserting the result of the movement determination ("approaching" or "moving away") into [movement determination result].

[0108] The hazard prediction device 3A performs the generation process in cooperation with the generation device 7, but the hazard prediction device 3A may also perform the generation process independently.

[0109] The hazard prediction system in the above-described embodiments (first and second embodiments) can be used not only to predict hazards in the monitoring area in real time, but also for the education of childcare workers. For example, by showing childcare workers images of the monitoring area, having them extract the hazards hidden there, and comparing them with the hazards predicted by the hazard prediction system, they can be educated to make accurate hazard predictions. When the hazard prediction system is used for educational purposes, the system may include the countermeasures established in the "establishment of countermeasures" phase of hazard prediction training and / or the behavioral goals set in the "setting of goals" phase as predicted hazard text in the basic hazard information. This allows the hazard prediction system to present not only the content of the hazard but also how to respond to that hazard, enabling more comprehensive education. Even when predicting hazards in the monitoring area in real time, the hazard prediction system may also present how to respond to the hazard.

[0110] While the hazard prediction system of the above embodiment predicts hazards in a nursery school, it can also predict hazards in various other locations such as workplaces in manufacturing and construction industries, company offices, and schools.

[0111] The above-described embodiment can be realized with a computer and memory configuration by using memory units 42, 52, and 72 as memory and implementing the processing of other components (excluding input means and display means) as a program as described above. Each component can also be realized with hardware such as a dedicated IC (Integrated Circuit) or FPGA (Field Programmable Gate Array). The information stored in each memory unit may be stored for a long period of time in storage (auxiliary storage device) in file format or the like. Furthermore, although the above-described embodiment is described in the form of a system or device, the present invention can also take the form of a method or program.

[0112] It should be noted that the present invention is not limited to the above embodiments, and various modifications are possible without departing from the spirit of the invention. Furthermore, matters not explicitly disclosed in the above embodiments do not deviate from what is normally practiced by those skilled in the art, and values ​​that can be easily anticipated by those skilled in the art may be adopted. [Explanation of symbols]

[0113] 1. 1A Hazard Prediction System 2. Imaging device 3.3A Hazard Prediction Device 4. Object detection device 5. Search device 6. User terminals 7 Generator 10 Networks 31, 31A Information Analysis Department 32, 32A Hazard Prediction Department 41, 51, 61, 71 Control Unit 42, 52, 72 storage section 62 Input means 63 Display means

Claims

1. A hazard prediction device that predicts dangers in a predetermined location, An information analysis unit acquires image information captured at the predetermined location and outputs detection information, which is information about the detected object detected from the image information. A hazard prediction device comprising: a hazard prediction unit that outputs hazard information, which is information relating to hazards foreseeable at a predetermined location, generated based on detection information using basic hazard information collected in advance as hazards foreseeable at the predetermined location.

2. The hazard prediction device according to claim 1, wherein the hazard prediction unit outputs information retrieved from the basic hazard information based on the name of the detected object included in the detection information as the hazard information.

3. The hazard prediction device according to claim 2, wherein the hazard prediction unit sets an accuracy for the element information constituting the hazard information according to the number of names of the related detected objects.

4. The information analysis unit outputs detection information, which also includes location information indicating the location of the detected object. The hazard prediction device according to claim 3, wherein the hazard prediction unit adjusts the accuracy according to the distance between detected objects determined from the position information.

5. The hazard prediction device according to claim 1, wherein the hazard prediction unit outputs information generated based on the detection information by a large-scale language model learned using the basic hazard information as the hazard information.

6. The information analysis unit outputs detection information, which also includes location information indicating the location of the detected object. The hazard prediction device according to claim 5, wherein the hazard prediction unit also uses information on the distance between detected objects obtained from the position information to output information generated based on the detection information as the hazard information.

7. The hazard prediction device according to claim 1, wherein the information analysis unit outputs detection information that distinguishes between adults and children when the detected object includes both adults and children.

8. The hazard prediction device according to claim 1, wherein the basic hazard information is comprised of information collected in hazard prediction training.

9. A hazard prediction system comprising a hazard prediction device and an imaging device according to any one of claims 1 to 8, The imaging device captures the predetermined location and outputs it as image information. A hazard prediction system characterized in that the information analysis unit acquires image information output from the imaging device.

10. The system will also be equipped with terminal devices for use by users. The hazard prediction system according to claim 9, wherein the terminal device inputs hazard information output by the hazard prediction unit.

11. The hazard prediction system according to claim 10, wherein the information analysis unit acquires image information output from the imaging device via the terminal device.

12. A hazard prediction method performed by a computer to predict hazards at a given location, An information analysis step which involves acquiring image information of the predetermined location, and acquiring or generating detection information which is information about the detected object detected from the image information, A hazard prediction method characterized by comprising: a hazard prediction step of outputting hazard information, which is information relating to hazards foreseeable at a predetermined location, generated based on the detection information using basic hazard information collected in advance as hazards foreseeable at the predetermined location.

13. The hazard prediction method according to claim 12, wherein in the hazard prediction step, information retrieved from the basic hazard information based on the name of the detected object included in the detection information is output as the hazard information.

14. The hazard prediction method according to claim 12, wherein in the hazard prediction step, information generated based on the detection information by a large-scale language model trained using the basic hazard information is output as the hazard information.

15. A hazard prediction program for causing a computer to perform the hazard prediction method described in any one of claims 12 to 14.

Images