Monitoring device, monitoring program, storage medium, and monitoring method

Abstract

To provide a monitoring device, a monitoring program, a storage medium, and a monitoring method for which a mobile monitoring device is used.SOLUTION: A monitoring device (MOS1) includes a mobile monitoring device (MM1) and a server (SV1). The mobile monitoring device (MM1) has an image capture unit (CM1) for capturing an image, a drive unit for moving its own device, an acquisition unit for acquiring a position of the mobile monitoring device, and a communication unit (COM) for transmitting monitoring information including the position of the mobile monitoring device and the image at the position to a server via wireless or wired communication means. The server (SV1) has the communication unit (COM) for receiving the monitoring information including the image at the position, and an output unit (OUT) for outputting an alarm.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a monitoring device, a monitoring program, a storage medium, and a monitoring method, and more particularly to a monitoring device, a monitoring program, a storage medium, and a monitoring method using a mobile monitoring device. [Background technology]

[0002] Livestock production has been rapidly expanding in recent years. Traditionally, most producers were family-run, but now agricultural cooperatives and corporations produce large amounts of chicken, eggs, pigs, cattle, and other livestock for domestic and international markets.

[0003] As livestock production became larger, production techniques and livestock breeding also progressed, and it became possible to steadily produce large quantities of high-quality livestock in standardized livestock barns. In particular, automated machines suitable for feeding and watering were put to practical use, and in large-scale livestock barns, almost no human labor was required except for cleaning.

[0004] However, there are also disadvantages to large-scale operations. In large-scale livestock farms, in order to ensure stable production of high-quality livestock, it is necessary to quickly detect and remove livestock that are in poor quality, such as sick or dead livestock. This requires experienced staff to concentrate and inspect the vast livestock farms, observing the condition of the livestock. However, there are only a few experienced staff, and inexperienced staff or staff who lack concentration may overlook or miss inspections, which could lead to the spread of diseases to other livestock or the outbreak of new diseases due to the decay of carcasses.

[0005] Surveillance cameras have been used for a long time, but with the spread of Internet lines, they are increasingly being connected to a network and monitored by a remote server. In addition, the number of pixels in cameras has increased, making it possible to analyze recorded images after the fact and identify accidents and crimes. The most common use of such surveillance cameras today is to install unmanned surveillance cameras in buildings and monitor the surveillance camera footage from remote locations to help with crime prevention. Security personnel actually observe the many images from the surveillance cameras on multiple display devices, recognize suspicious targets (suspicious people or objects), and, if necessary, dispatch security guards or warn the suspicious person.

[0006] As a conventional technology for automating a surveillance camera, there is an image processing device, an image processing system, and a program (see Patent Document 1). This is a technology that "sets a gaze area in a panoramic image of a surveillance area, analyzes the image of the set gaze area by an image analysis means to detect a moving object, and operates the zoom camera of the camera unit based on the detection to photograph the moving object. At that time, spatial relationship information with the panoramic image and time relationship information showing the transition of the gazed partial image on the time axis in the panoramic image are created for the gazed partial image, and the above relationship information is added to the gazed partial image and stored in the image storage unit. At the time of search, the above relationship information is used to search for the panoramic image and the gazed partial image, and both images are superimposed and displayed on the display unit." [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2010-233185 A Summary of the Invention [Problem to be solved by the invention]

[0008] The above conventional technology sets a monitoring area, and then zooms in and photographs any moving object within the monitoring area as a suspicious object. This is a technology that automatically tracks and zooms in on suspicious objects, but it does not allow for "automatic setting of the monitoring area." In addition, if the monitoring area is large or if multiple moving objects are present in the monitoring area, the processing will not be able to keep up unless a high-speed computing device is used, and there are disadvantages such as the inability to capture suspicious objects.

[0009] Thus, although the technology for automating surveillance cameras has advanced, no technology has been developed for identifying and identifying abnormal individuals in livestock or agricultural products.

[0010] SUMMARY OF THE PRESENT EMBODIMENT It is therefore an object of the present invention to provide a monitoring device, a monitoring program, a storage medium, and a monitoring method.

[0011] Another object of the present invention is to provide a monitoring device, a monitoring program, a storage medium, and a monitoring method using a mobile monitoring device. [Means for solving the problem]

[0012] In order to solve the above-mentioned problems, a monitoring device (such as a livestock barn monitoring device) according to a first aspect of the present invention comprises: An image acquisition unit that acquires an image including location information of the location where the image was taken; A memory unit for storing the acquired image and standard specific part information, which is standard physical information of a specific part of an individual (in the case of chickens, color information, dimensional information, shape information, etc. of a standard healthy chicken's comb, wattles, etc.); an identification unit that identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image; a warning unit which compares the identified specific part with standard specific part information stored in the storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, outputs warning information including location information of the individual with the specific part and an image including the specific part; Including, It is characterized by:

[0013] In addition, the monitoring device (livestock barn monitoring device) according to the second invention is an image acquisition unit that acquires an image including location information of the location where the image was taken; A memory unit for storing the acquired images and abnormal specific part information, which is physical information of specific parts of abnormal individuals (in the case of chickens, color information, dimensional information, shape information, etc. of the comb or wattle of an abnormal, sick, weakened, injured, or dead chicken (specific parts that tend to indicate a health condition are preferable)); an identification unit that identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image; a warning unit which compares the identified specific part with standard specific part information stored in the storage unit, and if the specific part and the abnormal specific part information are within a predetermined numerical value, or if the specific part and the abnormal specific part information match, outputs warning information including location information of the individual with the specific part and an image including the specific part; Including, It is characterized by:

[0014] In addition, the monitoring device (livestock barn monitoring device) according to the third invention is an image acquisition unit that acquires an image of an individual, the image including location information of the location where the image was taken; A storage unit for storing the acquired image; A generating unit that generates standard specific part information, which is standard physical information of at least one specific part of an individual based on each individual in the stored image (in the case of a chicken, color information, dimensional information, shape information, etc. of the chicken's comb, wattles, etc.); an identification unit that identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image; a warning unit which compares the identified specific part with the generated standard specific part information, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information differ, outputs warning information including location information of the individual with the specific part and an image including the specific part; Including, It is characterized by:

[0015] In addition, the monitoring device (livestock barn monitoring device) according to the fourth invention is The generation unit, Generate standard specific part information, which is standard physical information of at least one specific part of an individual, based on only images of each individual from a specified period (preferably the most recent 1-3 weeks) among the stored images. (In the case of chickens, color information, dimensional information, shape information, etc. of the chicken's comb, wattles, etc.) It is characterized by: Since each part of a livestock changes in shape, size, color, etc. as it grows, this configuration can reflect the standard condition of the current livestock, making it possible to more accurately identify abnormal individuals. In addition, the monitoring device (livestock barn monitoring device) according to the fifth invention is In the third or fourth invention, The generating unit, the identifying unit, or the warning unit, It operates based on AI techniques, It is characterized by:

[0016] In addition, the monitoring device (livestock barn monitoring device) according to the sixth invention is An image acquisition unit that acquires an image including location information of the location where the image was taken; A memory unit for storing the acquired images and an AI model trained using AI techniques (based on specific parts of the individual labeled in the previously acquired images); A recognition unit that uses the AI ​​model to recognize specific parts of the stored image (such as the comb and wattles in the case of chickens); a warning unit that uses the AI ​​model to determine whether the identified specific portion is abnormal, and if abnormal, outputs warning information including location information of the individual in the specific portion and an image including the specific portion; The present invention is characterized in that it includes:

[0017] In addition, the monitoring device (livestock barn monitoring device) according to the seventh invention is an image acquisition unit that acquires an image of an individual, the image including location information of the location where the image was taken; A storage unit for storing the acquired image; A generation unit that labels each individual in the stored images as abnormal (indicating unhealthy conditions such as illness, death, or weakness) or labels specific parts of the individual, and generates a trained AI model using AI techniques (e.g., techniques such as reinforcement learning or deep learning); an identification unit that identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image; a warning unit that uses the AI ​​model to determine whether the identified specific part is abnormal, and if it is abnormal, outputs (transmits) warning information including location information of the individual in the specific part and an image including the specific part; The present invention is characterized in that it includes:

[0018] In addition, the monitoring device (livestock barn monitoring device) according to the eighth invention is An image acquisition unit that acquires a video of an individual including location information of the location where the video was taken; A storage unit for storing the acquired video; A generation unit that labels the behavior of each individual in the stored video and generates a trained AI model using an AI technique (e.g., a technique such as reinforcement learning or deep learning); an identification unit that identifies the behavior of an individual in the stored video using an AI model; a warning unit that uses an AI model to determine whether the behavior of the identified individual is abnormal, and if abnormal, outputs warning information including location information of the individual and an image including the individual; The present invention is characterized in that it includes: Preferably, among the stored images, images from a specified period (preferably about the past 1-3 weeks) are weighted to generate an AI model. Since the shape, size, color, etc. of each part of livestock change as they grow, this configuration can reflect the standard condition of the current livestock, making it possible to more appropriately identify abnormal individuals. For example, in the case of pigs, it is possible to determine abnormalities by setting the pattern of spots that appear on the skin when they have a certain infection, the color, the degree of bloodshot eyes, etc.

[0019] In addition, a monitoring device (livestock barn monitoring device) according to a ninth aspect of the present invention comprises: a monitoring information acquisition unit that acquires location information of a monitored location and monitoring information including measurement values ​​(such as sound, odor, temperature, and humidity) measured at the monitored location; A storage unit for storing the acquired monitoring information; A generator that labels the measurements of the stored monitoring information and generates a trained AI model using an AI technique (e.g., a technique such as reinforcement learning or deep learning); an identification unit that identifies measurement values ​​of the stored monitoring information using the AI ​​model; a warning unit that uses the AI ​​model to determine whether the identified measurement value is abnormal, and if the identified measurement value is abnormal, outputs warning information including location information and the measurement information; The present invention is characterized in that it includes:

[0020] As described above, the solution of the present invention has been described as an apparatus, but the present invention can also be realized as a method, a program, or a storage medium having a program recorded thereon, which are substantially equivalent to the above, and it should be understood that these are also included within the scope of the present invention. Note that each step of the below-described method and program uses a processor such as a CPU or DSP for data processing as necessary, and stores input data, processed or generated data, etc. in a storage device such as a magnetic tape, HDD, or memory.

[0021] For example, a monitoring program according to a tenth aspect of the present invention, which is a program implementing the present invention, A monitoring program that causes one or more processors to function as the monitoring device according to any one of the first to ninth aspects of the present invention.

[0022] For example, a storage medium according to an eleventh aspect of the present invention is a storage medium, A computer-readable storage medium storing the monitoring program according to the tenth aspect of the present invention.

[0023] For example, a monitoring method using a monitoring device according to a twelfth aspect of the present invention, which realizes the present invention as a method, is as follows: An image acquisition step in which a computing device acquires an image including location information of the location where the image was taken; A storage step in which the computing device stores the acquired image and standard specific part information, which is standard physical information of a specific part of an individual (in the case of chickens, color information, shape information, etc. of a standard healthy chicken's comb, wattles, etc.); An identification step in which the computing device identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image; a warning step in which the computing device compares the identified specific part with standard specific part information stored in a storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information are different, outputs warning information including position information of the individual with the specific part and an image including the specific part; Including, It is characterized by:

[0024] A monitoring method according to a thirteenth aspect of the present invention comprises: An image acquisition step in which a computing device acquires an image including location information of a place where the image was taken; a storage step in which the computing device stores the acquired image and abnormal specific part information, which is physical information of a specific part of an abnormal individual (in the case of chickens, color information, shape information, etc. of the comb, wattles (specific parts that tend to indicate a health condition) of an abnormal, sick, weakened, injured, or dead chicken); An identification step in which the computing device identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image; a warning step in which the computing device compares the identified specific part with standard specific part information stored in a storage unit, and if the specific part and the abnormal specific part information are within a predetermined numerical value, or if the specific part and the abnormal specific part information match, outputs warning information including position information of the individual with the specific part and an image including the specific part; Including, It is characterized by:

[0025] A monitoring method according to a fourteenth aspect of the present invention comprises: An image acquisition step in which a computing device acquires an image of an individual, the image including location information of the location where the image was taken; A storage step in which the computing device stores the acquired image; A generation step in which the computing device generates standard specific part information, which is standard physical information of at least one specific part of an individual based on each individual in the stored image (in the case of a chicken, color information, shape information, etc. of the chicken's comb, wattles, etc.); An identification step in which the computing device identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image; a warning step in which the computing device compares the identified specific part with the generated standard specific part information, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information are different, outputs warning information including position information of the individual of the specific part and an image including the specific part; Including, It is characterized by:

[0026] A monitoring method according to a fifteenth aspect of the present invention comprises: In a monitoring method according to a thirteenth aspect of the present invention, The generating step includes: Generate standard specific part information, which is standard physical information of at least one specific part of an individual, based on only images of each individual from a specified period (preferably the most recent 1-3 weeks) among the stored images. (In the case of chickens, color information, dimensional information, shape information, etc. of the chicken's comb, wattles, etc.) It is characterized by: Since each part of a livestock changes in shape, size, color, etc. as it grows, this configuration can reflect the standard condition of the current livestock, making it possible to more accurately identify abnormal individuals.

[0027] A monitoring method according to a sixteenth aspect of the present invention comprises: In the fourteenth or fifteenth aspect of the present invention, The generating step, the identifying step, or the step It operates based on AI techniques, It is characterized by:

[0028] A monitoring method according to a seventeenth aspect of the present invention comprises: An image acquisition step in which a computing device acquires an image including location information of the location where the image was taken; A storage step in which the computing device stores the acquired image and an AI model trained using an AI technique based on the labeled specific part of the individual; An identification step in which the computing device identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image using the AI ​​model; a warning step in which the computing device uses the AI ​​model to determine whether the identified specific portion is abnormal, and if the specific portion is abnormal, outputs warning information including location information of the individual in the specific portion and an image including the specific portion; The present invention is characterized in that it includes:

[0029] A monitoring method according to an eighteenth aspect of the present invention comprises: An image acquisition step in which a computing device acquires an image of an individual, the image including location information of the location where the image was taken; A storage step in which the computing device stores the acquired image; A generation step in which the computing device labels specific parts of each individual in the stored images and generates a trained AI model using an AI technique (e.g., a technique such as reinforcement learning or deep learning); An identification step in which the computing device identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image using an AI model; a warning step in which the computing device uses an AI model to determine whether the identified specific portion is abnormal, and if abnormal, outputs (transmits) warning information including location information of the individual in the specific portion and an image including the specific portion; The present invention is characterized in that it includes:

[0030] A monitoring method according to a nineteenth aspect of the present invention comprises: An image acquisition step in which a computing device acquires an image of an individual, the image including location information of the location where the image was taken; a storage step in which the computing device stores the acquired image; A generation step in which the computing device labels the actions of each individual in the stored video and generates a trained AI model using an AI technique (e.g., a technique such as reinforcement learning or deep learning); An identification step in which the computing device identifies the behavior of an individual in the stored video using the AI ​​model; a warning step in which the computing device uses the AI ​​model to determine whether or not the behavior of the identified individual is abnormal, and if the behavior is abnormal, outputs warning information including location information of the individual and an image including the individual; The present invention is characterized in that it includes:

[0031] A monitoring program according to a twentieth aspect of the present invention, which is a program implementing the present invention, A monitoring program that causes a computer to execute the monitoring method according to any one of the eleventh to nineteenth aspects of the present invention.

[0032] In addition, a storage medium according to a 21st aspect of the present invention, which is a storage medium according to the present invention, is A computer-readable storage medium storing the monitoring program according to the twentieth aspect of the present invention.

[0033] As mentioned above, the above has been explained in terms of a monitoring device. Below, we will explain a typical mobile monitoring device, such as a drone or a self-propelled device using wheels or tracks, and an embodiment realized by distributed processing using a server.

[0034] A monitoring device according to a twenty-second aspect of the present invention comprises: A monitoring device including a mobile monitoring device and a server, The mobile monitoring device, An imaging unit that captures an image; A drive unit that moves the device itself; An acquisition unit (such as a GPS unit) that acquires the location of the mobile monitoring device; a communication unit that transmits monitoring information, including a position of the mobile monitoring device and an image at the position, to the server via a wireless or wired communication means; having The server, a communication unit that receives the monitoring information from the mobile monitoring device, the monitoring information including a position of the mobile monitoring device and an image at the position; A storage unit for storing the monitoring information, including the position of the mobile monitoring device and an image at that position, and standard specific body part information, which is standard physical information of a specific body part of an individual (in the case of chickens, color information, dimensional information, shape information, etc. of a standard healthy chicken's comb, wattles, etc.); an identification unit that identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image; a warning unit which compares the identified specific part with standard specific part information stored in the storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, outputs warning information including location information of the individual with the specific part and an image including the specific part; having It is characterized by: That is, the identified specific part is compared with the standard specific part information stored in the memory unit, and if it is determined that the identified specific part is abnormal (abnormal event (phenomenon)), warning information is output that includes location information of the individual with the specific part and an image including the specific part.

[0035] A monitoring device according to a twenty-third aspect of the present invention comprises: The drive unit of the mobile monitoring device is It is self-propelled, It is characterized by:

[0036] A monitoring device according to a twenty-fourth aspect of the present invention comprises: The generating unit, the identifying unit, or the warning unit, It operates based on AI techniques, It is characterized by:

[0037] In addition, a monitoring device (livestock barn monitoring device) according to a 25th aspect of the present invention comprises: A monitoring device including a mobile monitoring device and a server, The mobile monitoring device, An imaging unit that captures an image; A drive unit that moves the device itself; An acquisition unit (such as a GPS unit) that acquires the location of the mobile monitoring device; a communication unit that transmits monitoring information, including a position of the mobile monitoring device and an image at the position, to the server via a wireless or wired communication means; having The server, a communication unit that receives the monitoring information from the mobile monitoring device, the monitoring information including a position of the mobile monitoring device and an image at the position; A memory unit for storing the received images and an AI model trained using AI techniques (based on the specific parts of the individual labeled in the previously acquired images); A recognition unit that uses the AI ​​model to recognize specific parts of the stored image (such as the comb and wattles in the case of chickens); a warning unit that uses the AI ​​model to determine whether the identified specific portion is abnormal, and if abnormal, outputs warning information including location information of the individual in the specific portion and an image including the specific portion; The present invention is characterized in that it includes: The server, a memory unit for storing a number of images and monitored events associated with at least some of the images; The monitoring unit, When monitoring whether a set monitoring event has occurred, an AI technique is used to refer to the storage unit and monitor whether the set monitoring event has occurred. It is characterized by:

[0038] A monitoring device according to a twenty-sixth aspect of the present invention comprises: The communication unit of the mobile monitoring device Further receiving monitoring event salience information (feeding information (feeding location, time, etc.), intruder intimidation information (location, time, etc. of alarm sound generation), etc.), The drive unit of the mobile monitoring device is Controlling the movement of the device itself based on the monitored event salience information. It is characterized by:

[0039] A monitoring device according to a 27th aspect of the present invention comprises: The communication unit of the server Further receiving monitoring event salience information (feeding information (feeding location, time, etc.), intruder intimidation information (location, time, etc. of alarm sound generation), etc.), The server, based on the monitoring event salience information, controlling the drive unit of the mobile monitoring device via the communication unit to move the mobile monitoring device; It is characterized by:

[0040] A monitoring device according to a twenty-eighth aspect of the present invention comprises: The monitored event salience information is One or more items selected from the group consisting of feeding information (location and time of feeding, etc.), intruder scare information (location and time of warning light, warning vibration, warning sound, etc.), odor information (odors that the monitored chickens dislike or are excited by), light information (illuminance of lighting and indoors, flashing lights, rotating lights, etc.), opening / closing information (opening / closing of windows and ventilation fans, etc.), and air conditioning setting information. It is characterized by:

[0041] A monitoring device according to a twenty-ninth aspect of the present invention comprises: The mobile monitoring device, The device further includes a monitoring event conspicuous situation generating means for generating the situation of the monitoring event conspicuous information (for example, a rotating light that generates light, a speaker that generates sound, a rod-shaped part that makes vibrations and sounds by contacting the chicken coop as it moves, a signal output part that starts feeding, a control signal output part that controls the output of lighting and sound, etc.), It is characterized by:

[0042] A monitoring program according to a 30th aspect of the present invention comprises: A monitoring program that causes one or more arithmetic processing devices to function as the monitoring device according to any one of the twenty-first to twenty-eighth aspects of the present invention.

[0043] A storage medium according to a thirty-first aspect of the present invention comprises: A computer-readable storage medium storing the monitoring program according to the thirtieth aspect of the present invention. [Brief description of the drawings]

[0044] [Figure 1] FIG. 1 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a flowchart illustrating an example of a process executed by the monitoring device illustrated in FIG. [Diagram 3] FIG. 3 is a schematic diagram illustrating the monitoring technique of the mobile monitoring device of the present invention. [Figure 4] FIG. 4 is a schematic diagram illustrating another monitoring technique of the mobile monitoring device of the present invention. [Diagram 5] FIG. 5 is a schematic diagram illustrating the function of the acquisition unit in the present device. [Figure 6] FIG. 6 is a schematic diagram for explaining the flight route setting function of this device. [Figure 7] FIG. 7 is a schematic diagram for explaining a monitoring technique by the monitoring unit. [Figure 8] FIG. 8 is a schematic diagram for explaining map-like alarm information. [Figure 9] FIG. 9 is a schematic diagram for explaining map-like alarm information. [Figure 10] FIG. 10 is a schematic diagram illustrating the monitoring technique of the mobile monitoring device of the present invention when applied to a chicken coop. [Figure 11] FIG. 11 is a schematic diagram for explaining a monitoring technique of the mobile monitoring device of the present device provided with a monitoring event prominent situation generating means. [Figure 12] FIG. 12 is a schematic diagram illustrating the monitoring technique of the mobile monitoring device of the present invention when applied to a chicken coop. [Figure 13] FIG. 13 is a schematic diagram illustrating the monitoring technique of the mobile monitoring device of the present invention when applied to a chicken coop. [Figure 14] FIG. 14 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. [Figure 15] FIG. 15 is a flowchart showing an example of a process executed by the monitoring device shown in FIG. [Figure 16] FIG. 16 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (generation of standard specific part information). [Figure 17] FIG. 17 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (utilization of standard specific part information). [Figure 18] FIG. 18 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (AI model generation). [Figure 19] FIG. 19 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (utilizing an AI model). [Figure 20] FIG. 20 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (application of AI model generation). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0045] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0046] <Embodiment 1> Fig. 1 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. As shown in the figure, monitoring device MOS1 includes mobile monitoring devices MM1 and MM2 and a server SV1. Server SV1 has a control unit (CPU, arithmetic processing unit, processor) CON, an input unit IN, an output unit OUT, a communication unit COM, a memory unit MEM, and a display unit DIS. Unless otherwise specified, mobile monitoring device MM2 has the same configuration and functions as mobile monitoring device MM1, but for convenience of drawing and explanation, some components have been omitted.

[0047] The mobile monitoring device MM1 has cameras CM1-3 as an imaging unit, a sensor SNSR (barometer, gyro sensor, etc.), a control unit CON (CPU, arithmetic processing unit, processor), a communication unit COM, a monitoring unit MON (this function may be provided on the server side), an output unit OUT, a memory unit STR, and a GPS unit GR, and may further have an input unit (not shown) and a display unit (not shown). It is preferable to provide two cameras to photograph a monitoring target such as a chicken coop and one to photograph a position indicator (such as a point-like marker or a line-like one) installed on a ceiling or floor, but one or two cameras may serve both functions. It is also preferable to use a high-performance model that can photograph with little environmental light even at night. Furthermore, the camera may be provided with a separate infrared detector or the above-mentioned camera may be equipped with an infrared function to detect the temperature of the monitoring target and use it as a coloring material to determine the condition of the target, for example, whether it is dead or sick.

[0048] The mobile monitoring device MM1 has, as sensors SNSR, a position measurement unit (not shown, which may be a GPS unit) that measures the position of the device itself, a gyro sensor that measures angular velocity, an acceleration sensor that measures acceleration, an altitude sensor that measures altitude, and a rangefinder that measures or estimates the distance traveled by the device itself.

[0049] The mobile monitoring device MM1 is an imaging unit CM1-3 for capturing an image; A drive unit DR that moves the device itself; an acquisition unit RET (such as a GPS unit) for acquiring the location of the mobile monitoring device; a communication unit COM that transmits monitoring information, including a position of the mobile monitoring device and an image at the position, to the server via a wireless or wired communication means; has.

[0050] The server SV1 The server, A communication unit COM (and an acquisition unit ACQ) that receives (acquires) the monitoring information from the mobile monitoring device, the monitoring information including the position of the mobile monitoring device and an image at that position; a memory unit MEM for storing the monitoring information MI, including the position of the mobile monitoring device and an image at that position, and standard specific part information SPI, which is standard physical information of a specific part of an individual (in the case of a chicken, color information, dimensional information, shape information, etc. of a standard healthy chicken's comb, wattles, etc.); An identification unit IDP that identifies specific parts of the stored image (such as the comb and wattles in the case of chickens) a warning unit WAN which compares the identified specific part with standard specific part information stored in a storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information differ, outputs warning information including location information of the individual with the specific part and an image including the specific part; has. For example, it is preferable to express color information of a healthy chicken's comb, wattles, etc., in RGB values. Below is the one with the strongest red hue. RGB (255.0.0) It is preferable to set the RGB values ​​within a range that is healthy and standard and acceptable, and it is preferable to set the RGB values ​​based on statistical information of the livestock barn. In addition, as for the size information, since individuals that cannot grow to a standard size (dimensions) are likely to die due to illness, weakness, or other factors, it is important in terms of an appropriate breeding environment to be able to grasp such abnormalities.

[0051] The output alarm is preferably an email transmission, an audio warning output or transmission, an image output, a warning to a security guard, a warning to an administrator, a report to a security company or the police, etc. In addition, when monitoring whether a set monitoring event has occurred, the monitoring unit preferably uses an AI technique to refer to the image information IM in the storage unit (or derived information derived from the image information (such as a characteristic pattern of the image)) and monitors whether the set monitoring event has occurred.

[0052] The mobile monitoring device MM1, The guide portion GD is movably supported, placed or suspended on the guide rail and guided along the guide rail; The drive unit DR is The device itself is moved along the guide rail while being supported, placed or suspended on the guide rail.

[0053] CAD information of the building, shelves, chicken coop, etc. to be monitored may be acquired from a building information server or memory unit MEM connected via a network NET. A movement route is set from the CAD information, position index arrangement information that corresponds to or is made to correspond to the spatial coordinate system of this CAD information, and layout information of the objects to be monitored that are arranged in the building (chicken coop cages, desks, chairs, equipment, etc.). The display unit DIS can display information stored in the device and information generated.

[0054] Each functional unit included in the control unit of the server SV1 is preferably realized by a program module loaded into the memory space of the control unit. In addition, when the mobile device is realized as a drone or a self-propelled device, each functional unit can be realized by a program module loaded into the memory space of the control unit, or each functional unit can be realized by a built-in GPS unit or camera unit. Usually, the server downloads software that causes the processor to function as each unit of the device from a storage unit or a website, installs the software on the PC, and starts it, so that the PC operates as the device. Note that each functional unit provided in the control unit is merely a set of steps that are functionally consistent, and it is possible to combine multiple functional units into one functional unit, incorporate a part of it into another functional unit, or divide it into another independent functional unit.

[0055] It should be noted that the generated / extracted information, intermediate data, and acquired data can be transmitted to the outside, displayed on the display unit, and stored in the storage unit in the other embodiments described below. It is preferable that the present device is constructed on a computer such as a general-purpose computer, a specific-purpose computer, a server, a PC, or a mobile terminal such as a smartphone, or that a program module that realizes (executes) the functions and processing procedures (methods) of the present device on a computer is stored in a CPU or storage unit of the computer, or is read from an external server or storage, and this also applies to each of the following embodiments. In addition, each functional unit may be distributed to a separate computer or device connected by a network. In addition, a plurality of functional units may be integrated into one, or some of the processing steps may be assigned to another functional unit. Furthermore, in this embodiment, the server and the mobile monitoring device are defined as separate entities, but the mobile monitoring device may include some or all of the functions of the server.

[0056] The server SV1 has a generation unit GEN that generates standard specific part information (in the case of chickens, color information, dimensional information, shape information, etc. of the chicken's comb, wattles, etc.) which is standard physical information of at least one specific part of the individual based on each individual in the stored image. This makes it possible to obtain reference information that matches the documents of the livestock house and livestock, and to more appropriately identify, specify, and extract abnormal individuals and issue an alarm.

[0057] Fig. 16 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (generation of standard specific part information). As shown in the figure, standard specific part information SPI1 (which is numerical information on the color of the comb, preferably specified in RGB) and standard specific part information SPI2 (which is numerical information on the size of the wattle) are generated from image information SI3 showing chickens BD1-BD7 and stored in a memory unit. The next figure shows how to utilize the standard specific part information.

[0058] 17 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (utilizing standard specific body part information). As shown in the figure, the standard specific body part information SPI1 is compared with the color of the comb of chicken BD12. In this case, the color of the comb of chicken BD12 falls within the range of the standard specific body part information SPI1, so it is determined that they match and chicken BD12 is determined to be normal. Then, another individual is compared.

[0059] On the other hand, the standard specific part information SPI1 is compared with the color of the comb of chicken BD17. In this case, the color of the comb of chicken BD17 is outside the range of the standard specific part information SPI1, so it is determined that there is no match, that is, there is an abnormality, and warning information including the position and image of individual BD17 is output.

[0060] Fig. 2 is a flow chart showing an example of processing executed by the monitoring device shown in Fig. 1. The monitoring device performs processing in cooperation with two devices, a server SV1 and a mobile monitoring device MM1. If information and functions in other servers are installed in the mobile device, it may be possible to eliminate the need for cooperation with the server.

[0061] In step S11, the camera of the mobile monitoring device captures an image. Next, in step S12, the acquisition unit acquires the position of the mobile monitoring device. Then, in step S13, monitoring information including the position of the mobile monitoring device and an image at that position is transmitted to the server via wireless or wired communication means. In step S14, the server causes the driving unit of the mobile monitoring device to move the mobile monitoring device along the guide rail while confirming the position using a position indicator and / or a gyro sensor and an acceleration sensor (or GPS, rangefinder). Note that the movement may be a simple method such as supplying motor power for moving a predetermined distance (for example, the distance of one cage in a chicken coop) without using a sensor or operating the motor for a predetermined time.

[0062] In step S15, the camera of the mobile monitoring device captures an image. Next, the position of the mobile monitoring device is acquired, and in step S16, monitoring information including the position of the mobile monitoring device and an image at that position is transmitted to a server via wireless or wired communication means. In step S17, it is determined whether the end of the rail is reached or whether the remaining power of the battery mounted on the mobile monitoring device is insufficient. If the determination conditions are met, the process proceeds to step S19-2 and the device returns to the base. If the determination conditions are not met, the process proceeds to step S18.

[0063] In step S18, it is determined whether an abnormal event (monitoring event) has occurred in the image. If an abnormal event has not occurred, the process returns to step S15 and the movement and imaging are continued. If an abnormal event has occurred, an alarm may be issued as is, but in order to reduce false recognition, it is preferable to proceed to step S18-1, stop or slow down the mobile monitoring device, and continue observing the image of the abnormal event. Then, if it is determined in step S18-2 that the occurrence of the abnormal event is continuing, an alarm including the position where the abnormal event occurred and the image (including video) at that time is output in step S19. If it is determined in step S18-2 that the occurrence of the abnormal event is not continuing, it is determined that the occurrence is a false recognition, and no alarm is issued, and the process returns to step S15 and the movement (monitoring) is continued. After issuing the alarm in step S19, the process proceeds to step S19-1, and it is determined whether the rail (route) has ended or not, or whether the remaining power of the onboard battery is insufficient, as in step S17. If the determination condition is met, the mobile monitoring device returns to its base (charging spot). If the determination conditions are not met, the process returns to step S15 and movement (monitoring) continues.

[0064] The abnormal event is determined, for example, by comparing the identified specific part with the standard specific part information stored in the storage unit, and if the specific part and the standard specific part information are apart by a predetermined value or more, or if the specific part and the standard specific part information differ, warning information including position information of the individual of the specific part and an image including the specific part is output. That is, the identified specific part is compared with the standard specific part information stored in the storage unit, and if it is determined that the identified specific part is abnormal (abnormal event (phenomenon)), warning information including position information of the individual of the specific part and an image including the specific part is output.

[0065] FIG. 3 is a schematic diagram for explaining the monitoring technique of the mobile monitoring device of the present device. As shown in the figure, the position indices PI1 to 4 installed on the ceiling of the building BLD are photographed by the camera CM3, and the position of the device itself is recognized based on the indices. The position indices are arranged at the coordinates of the position index arrangement information, and the mobile monitoring device MM1 controls the movement (drive) of the device itself while recognizing the position of the device itself using the position indices and sequentially recognizing the position indices PI1 to 4 along the moving route RT1. When moving, the device moves along a guide rail GR (which may be placed or installed on the ceiling or wall) placed on the floor. The position index may be integrated into the guide rail GR. Also, the position may be measured and estimated using a range finder or the like without using the position index. Also, the position index is preferably one or more of fluorescent materials, letters, figures, patterns, colors, etc., and is preferably photographed by the camera CM3 facing the ceiling and the position index is preferably image-recognized. Each position index is unique, and by individually recognizing them and referring to the position index arrangement information LILI, it is possible to determine where the mobile monitoring device is located. Alternatively, the location indicator may be a device that emits a specific radio or optical signal, a WIFI base station, or a wireless tag (in this case, the signal is read by a wireless communication unit, optical receiver, etc., instead of a camera). It is also possible to provide an indoor GPS signal unit instead of the location indicator and have the GPS unit of the mobile monitoring device read this.

[0066] While moving, the mobile monitoring device photographs the left and right monitoring target shelves SBJ1, SBJ2 (installed in the building BLD) with the cameras CM1, CM2, and monitors whether or not a monitoring event has occurred there.

[0067] FIG. 4 is a schematic diagram for explaining another monitoring technique of the mobile monitoring device of the present device. As shown in the figure, the mobile monitoring device MM2 is self-propelled, driven by tires or caterpillars, and moves along a guide line GL placed on the floor or painted on it. Other functions are the same as those of FIG. 16. The guide line GL is preferably a magnetic tape, a coating impregnated with magnetism, a film, tape, coating with a distinctive color scheme (e.g., white, yellow, etc.), etc. The guide line corresponds to the flight route described above, and functions as a travel route.

[0068] FIG. 5 is a schematic diagram for explaining the function of the acquisition unit in the device. As shown in the figure, the device acquires CAD information CI of a building. This CAD information is a three-dimensional CAD, and the building's internal components, walls, ceilings, floors, etc. have three-dimensional spatial coordinates. The device also acquires layout information SLI of monitored installations (chicken cages, desks, chairs, equipment, etc.) arranged in the building, corresponding to the spatial coordinate system of the CAD information CI of the building. The device incorporates (merges) the layout information SLI into the CAD information CI. Then, the device acquires position index arrangement information PI1-4 corresponding to the spatial coordinate system of the CAD information CI of the building, and incorporates (merges) the position index arrangement information PI1-4 into the CAD information CI, as shown in the lower part of the figure. As a result, the device obtains CAD information CIALL in which the internal structure of the building, in which the position indexes and installations are arranged, is described in its entirety. The device sets a flight route and a movement route in this CAD information CIALL, which will be explained in the next figure.

[0069] Here, the mobile device is described as a drone, but of course, the mobile device may be a self-propelled vehicle. FIG. 6 is a schematic diagram for explaining the function of setting a flight route in this device. As shown in the figure, it is assumed that there are monitoring targets SBJ1 and SBJ2, which are installations arranged in accordance with the layout information SLI in the CAD information CIALL, in which the position of the drone base DB is also described, and there is an aisle (a space suitable for flight) between them. This device automatically sets a flight route FR1 based on a route setting rule (stored in the memory unit) such that the flight route is set in a space of a predetermined size where the monitoring targets can be monitored (it can also be set manually). Similarly, it is assumed that there are monitoring targets SBJ3 and SBJ4, and there is an aisle (a space suitable for flight) between them. This device has a function of recognizing a space large enough for a person to pass through as an aisle. Based on a route setting rule (stored in the memory unit) such that the flight route is set in a space of a predetermined size where the monitoring targets can be monitored, at a predetermined height (usually 160 cm, which is about the eye level of the average height), the flight route FR2 is also automatically set (it can also be set manually). As shown in the figure, when flight routes FR1 and FR2 are set, the flight routes are set in the order of FR1 first and FR2 second. Alternatively, all flight routes may be linked together to set one flight route.

[0070] In this example, the target to be monitored is assumed to be a one-tiered chicken coop, but in the case of two or three tiers, the flight route will be the same coordinates on a plane, but it is possible to set a different flight route with successively changing altitudes to monitor the second, third, etc. tiers. Alternatively, as in Figure 1, cameras that capture images of each tier may be installed in the same number as the number of tiers.

[0071] FIG. 7 is a schematic diagram for explaining the monitoring technique. An example of monitoring a chicken coop will be explained. As shown in the figure, chickens BD1 to BD7 are present on the screen interface SL1 captured by the camera. This device has logic for recognizing specific parts of chickens, such as the comb and beak, by shapes, colors, patterns, etc., and captures all chickens photographed on the screen. Then, it observes the movements of each individual chicken and extracts those that are motionless. In addition, it compares the specific parts with standard specific part information, which is standard physical information of the specific parts of an individual chicken, which is a master stored in the memory unit (in the case of chickens, color information, dimensional information, shape information, etc. of the comb, wattles, etc. of a standard healthy chicken), and if the specific parts and the standard specific part information are different by a predetermined value or more, or if the specific parts and the standard specific part information are different, that is, whether there is an abnormality or not is judged. The screen interface SL2 shows that the comb HDT4 of the chicken BD4 is whitish, which is a color tone quite different from the bright red color of the comb of a normal healthy chicken, which is the standard specific body part information, and therefore is determined to be abnormal. Such chickens and chickens that are not moving are determined to be monitoring events, that is, abnormal events (death, illness, or emaciation). The chicken BD4 on the screen interface SL3 is a chicken that has been determined to be an abnormal event, and an alarm is issued at this point (or when it returns to the base). In addition, in the case of chickens, since their bodies are covered with feathers, it is difficult to measure their body temperature from the outside using normal methods. However, the comb part has no feathers, and blood vessels run just below it, so that the body temperature can be easily measured by measuring the temperature of this part with an infrared camera or the infrared function of a normal camera, and it is possible to reliably determine whether the chicken is dead or not. This device captures the comb as a region of interest using a known image recognition technique, and furthermore, by measuring the temperature of the comb, it is possible to more accurately determine abnormal events.

[0072] In this manner, in this embodiment, it is possible to identify dead, sick, or weak chickens by checking specific parts of the chicken, such as the comb and the maroon color of the wattles.

[0073] FIG. 8 is a schematic diagram for explaining map-like alarm information. Images and videos of the occurrence of a monitoring event and the coordinates of the location are transmitted (output) from a drone or a mobile device. This makes it difficult for the staff who received the alarm to grasp the situation, so the device issues map-like alarm information WRN1 that displays the abnormal event occurrence points OABE1-3 of the buildings BLD1 and 2 where the monitoring event (abnormal event) occurred, as well as detailed information such as "Abnormal event occurred (3rd row), cage number A101BR031." This alarm is a flattening of CAD information to display the abnormal event occurrence points. Based on the alarm information, the staff checks the cage and performs the prescribed work (in this example, the work of removing dead or weakened individuals (chickens)).

[0074] Figure 9 is a schematic diagram explaining map-like alarm information. Map-like alarm information WRN1 is a plan view, but in a chicken coop or the like, there are several cage shelves in the vertical direction, and it is unclear at what height on the plane the cage shelves are. Map-like alarm information WRN2 in Figure 13 corresponds to this. This is used for the purpose of allowing an operator to look at the map in Figure 8, and then reach the point where the abnormal event occurred, and then look at the map in Figure 9 to understand at what vertical height of the shelf the abnormality occurred.

[0075] FIG. 10 is a schematic diagram illustrating the monitoring technique of the mobile monitoring device of this device when applied to a chicken coop. As shown in the figure, the guide section GD of the mobile monitoring device MM3 is supported on a guide rail GR installed on the ceiling SL of the building BLD, and the mobile monitoring device MM3 is suspended. The mobile monitoring device MM3 controls the movement (drive) of its own device along a movement route RT1 while recognizing its own position using some kind of position indicator or a range finder. When moving, it moves along the guide rail GR installed on the ceiling SL. The position indicator may be integrated into the guide rail GR. Also, the position may be measured and estimated using a range finder or the like without using a position indicator.

[0076] The mobile monitoring device MM3 has cameras CM2-3 for the upper chicken coops CG31, CG41, and cameras CM4-5 for the lower chicken coops CG32, CG42. In this example, dead or sick or otherwise weak chickens are abnormal events, while chickens BD11, BD12 are normal and are not recognized as abnormal events. Chickens BD13, BD14 are crouching or fallen, so the server determines this to be an abnormal event and issues an alarm. When the alarm is issued, the chicken coop number CG32 or CG42 indicating the location, or some other location information, and the image are output.

[0077] By using the monitoring event salience information, it is possible to improve the accuracy of monitoring and reduce false alarms to a considerable extent. The monitoring event salience information is one or more selected from the group consisting of feeding information (place and time of feeding, etc.), intruder scare information (location and time of warning light, warning vibration, warning sound generation, etc.), odor information (odor that the monitored chickens dislike or are excited by, etc.), light information (illuminance of lighting and indoors, flashing, rotating lights, etc.), opening / closing information (opening / closing of windows and ventilation fans, etc.), and air conditioning setting information.

[0078] Therefore, it is preferable to further provide the mobile monitoring device with a monitoring event salience situation generating means for generating the situation of the monitoring event salience information (for example, a rotating light that emits light, a speaker that emits sound, a rod-shaped part that comes into contact with the chicken coop as it moves and produces vibrations and sound, a signal output unit that starts feeding, a control signal output unit that controls the output of lighting and sound, etc.).

[0079] FIG. 11 is a schematic diagram for explaining the monitoring technique of the mobile monitoring device of the present device provided with a monitoring event conspicuous situation generating means. As shown in the figure, the mobile monitoring device MM4 has, as a monitoring event conspicuous situation generating means, rotating lights PL1, PL2 (rotating lights) that generate light (preferably light, color, or wavelength of light that does not stress the chickens and activates them) only in the area to be monitored to stimulate the chickens, and chicken coop striking rods BR1, BR2 that generate vibrations and sounds only in the area to be monitored to stimulate the chickens. It is preferable that the vibrations and sounds at this time are as close as possible to the sound of feeding and the vibrations of feeding. This is because when healthy chickens are given the sound and vibrations of feeding, they will always react, stand up, and become active. In addition, a speaker that reproduces and plays the sound of feeding is suitable as a monitoring event conspicuous situation generating means. Alternatively, a means for generating the smell of food or the smell of a male (pheromone) is also effective as a monitoring event conspicuous situation generating means.

[0080] FIG. 12 is a schematic diagram explaining the monitoring technique of the mobile monitoring device of this device when applied to a chicken coop. As shown in the figure, a line reading sensor LSN of the mobile monitoring device MM5 reads a guideline GL that is installed, laid or painted on the floor FL of the building, and the mobile monitoring device MM5 automatically moves along this guideline GL. The mobile monitoring device MM5 recognizes its own position using the guideline (preferably in combination with a rangefinder or gyro sensor, etc.) and controls the movement (drive) of its own device along the guideline GL. The guideline may not only be used for guidance, but may also be integrated with a position indicator. Also, the position may be measured and estimated using a rangefinder, etc., without using a position indicator.

[0081] FIG. 13 is a schematic diagram for explaining the monitoring technique of the mobile monitoring device of the present device when applied to a chicken coop. As shown in the figure, a line reading sensor LSN installed on the top of the mobile monitoring device MM6 reads a guideline GL installed, laid, or painted on the ceiling SL of the building, and automatically moves along this guideline GL. The mobile monitoring device MM5 controls the movement (drive) of its own device along the guideline GL while recognizing its own device's position by the guideline (preferably using a range finder, an accelerometer, and / or a gyro sensor in combination). The guideline may be one that not only guides but also integrates a position indicator. Also, the position may be measured and estimated by a range finder or the like without using a position indicator. The advantage of this embodiment is that it can prevent the guideline installed on the floor from being read incorrectly when it is soiled with chicken droppings or feathers in a chicken coop, etc. This is because there is almost no possibility that the ceiling will be contaminated by chicken droppings or feathers.

[0082] The above-mentioned monitoring technique uses image information as a monitoring method, but a monitoring technique using a distance sensor will be described in the following embodiment 3.

[0083] <Embodiment 2> Fig. 14 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. As shown in the figure, monitoring device MOS1 includes mobile monitoring devices MM1 and MM2 and a server SV2. Server SV2 has a control unit (CPU, arithmetic processing unit, processor) CON, an input unit IN, an output unit OUT, a communication unit COM, a memory unit MEM, and a display unit DIS. Unless otherwise specified, mobile monitoring device MM2 has the same configuration and functions as mobile monitoring device MM1, but for convenience of drawing and explanation, some components have been omitted.

[0084] Also, server SV2 is an image acquisition unit that acquires an image of an individual, the image including location information of the location where the image was taken; A memory unit MEM for storing the acquired image; A generation unit GEN that labels specific parts of each individual in the stored images and generates an AI model AIM (stored in the memory unit as an AI algorithm and various derived data) trained using an AI technique (e.g., a technique such as reinforcement learning or deep learning); An identification unit IDP that uses the AI ​​model to identify specific parts in the stored image (such as the comb and wattles in the case of chickens); a warning unit WAN that uses the AI ​​model to determine whether the identified specific part is abnormal, and if it is abnormal, outputs (transmits) warning information including location information of the individual with the specific part and an image including the specific part; Includes.

[0085] The present invention can also be implemented in a form in which the motion of an individual is identified and an abnormality is determined. Server SV2 is A generation unit GEN that labels the behavior of each individual in the stored video and generates a trained AI model using an AI technique (e.g., a technique such as reinforcement learning or deep learning); An identification unit IDP that identifies the behavior of an individual in the stored video using an AI model; A warning unit WAN that uses an AI model to determine whether the behavior of the identified individual is abnormal, and if abnormal, outputs warning information including location information of the individual and an image including the individual; Includes.

[0086] The mobile monitoring device MM1 has cameras CM1-3 as an imaging unit, a sensor SNSR (barometer, gyro sensor, etc.), a control unit CON (CPU, arithmetic processing unit, processor), a communication unit COM, a monitoring unit MON (this function may be provided on the server side), an output unit OUT, a memory unit STR, and a GPS unit GR, and may further have an input unit (not shown) and a display unit (not shown). It is preferable to provide two cameras to photograph a monitoring target such as a chicken coop and one to photograph a position indicator (such as a point-like marker or a line-like one) installed on a ceiling or floor, but one or two cameras may serve both functions. It is also preferable to use a high-performance model that can photograph with little environmental light even at night. Furthermore, the camera may be provided with a separate infrared detector or the above-mentioned camera may be equipped with an infrared function to detect the temperature of the monitoring target and use it as a coloring material to determine the condition of the target, for example, whether it is dead or sick.

[0087] The mobile monitoring device MM1 has, as sensors SNSR, a position measurement unit (not shown, which may be a GPS unit) that measures the position of the device itself, a gyro sensor that measures angular velocity, an acceleration sensor that measures acceleration, an altitude sensor that measures altitude, and a rangefinder that measures or estimates the distance traveled by the device itself.

[0088] The mobile monitoring device MM1 is an imaging unit CM1-3 for capturing an image; A drive unit DR that moves the device itself; an acquisition unit RET (such as a GPS unit) for acquiring the location of the mobile monitoring device; a communication unit COM that transmits monitoring information, including a position of the mobile monitoring device and an image at the position, to the server via a wireless or wired communication means; has.

[0089] The server SV1 The server, A communication unit COM (and an acquisition unit ACQ) that receives (acquires) the monitoring information from the mobile monitoring device, the monitoring information including the position of the mobile monitoring device and an image at that position; a memory unit MEM for storing the monitoring information MI, including the position of the mobile monitoring device and an image at that position, and standard specific part information SPI, which is standard physical information of a specific part of an individual (in the case of a chicken, color information, dimensional information, shape information, etc. of a standard healthy chicken's comb, wattles, etc.); An identification unit IDP that identifies specific parts of the stored image (such as the comb and wattles in the case of chickens) a warning unit WAN which compares the identified specific part with standard specific part information stored in a storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information differ, outputs warning information including location information of the individual with the specific part and an image including the specific part; has.

[0090] The output alarm is preferably an email transmission, an audio warning output or transmission, an image output, a warning to a security guard, a warning to an administrator, a report to a security company or the police, etc. In addition, when monitoring whether a set monitoring event has occurred, the monitoring unit preferably uses an AI technique to refer to the image information IM in the storage unit (or derived information derived from the image information (such as a characteristic pattern of the image)) and monitors whether the set monitoring event has occurred.

[0091] The mobile monitoring device MM1, The guide portion GD is movably supported, placed or suspended on the guide rail and guided along the guide rail; The drive unit DR is The device itself is moved along the guide rail while being supported, placed or suspended on the guide rail.

[0092] CAD information of the building, shelves, chicken coop, etc. to be monitored may be acquired from a building information server or memory unit MEM connected via a network NET. A movement route is set from the CAD information, position index arrangement information that corresponds to or is made to correspond to the spatial coordinate system of this CAD information, and layout information of the objects to be monitored that are arranged in the building (chicken coop cages, desks, chairs, equipment, etc.). The display unit DIS can display information stored in the device and information generated.

[0093] Each functional unit included in the control unit of the server SV1 is preferably realized by a program module loaded into the memory space of the control unit. In addition, when the mobile device is realized as a drone or a self-propelled device, each functional unit can be realized by a program module loaded into the memory space of the control unit, or each functional unit can be realized by a built-in GPS unit or camera unit. Usually, the server downloads software that causes the processor to function as each unit of the device from a storage unit or a website, installs the software on the PC, and starts it, so that the PC operates as the device. Note that each functional unit provided in the control unit is merely a set of steps that are functionally consistent, and it is possible to combine multiple functional units into one functional unit, incorporate a part of it into another functional unit, or divide it into another independent functional unit.

[0094] Fig. 15 is a flow chart showing an example of processing executed by the monitoring device shown in Fig. 1. The monitoring device performs processing in cooperation with two devices, a server SV1 and a mobile monitoring device MM1. If information or functions in other servers are installed in the mobile device, it may be possible to eliminate the need for cooperation with the server.

[0095] The generation unit preliminarily labels specific parts of each individual in the stored image, generates an AI model trained using an AI technique (e.g., a technique such as reinforcement learning or deep learning), and stores the model in the storage unit. Then, in step S21, the camera of the mobile monitoring device captures an image. Next, in step S22, the acquisition unit acquires the position of the mobile monitoring device. Then, in step S23, the server transmits monitoring information including the position of the mobile monitoring device and an image at the position to the server via wireless or wired communication means. In step S24, the server moves the mobile monitoring device along the guide rail while the drive unit of the mobile monitoring device confirms the position using a position indicator and / or a gyro sensor and an acceleration sensor (or GPS, rangefinder). Note that the movement may be a simple method such as supplying motor power for moving a predetermined distance (e.g., the distance of one cage in a chicken coop) without using a sensor or operating the motor for a predetermined time.

[0096] In step S25, the camera of the mobile monitoring device captures an image. Next, the position of the mobile monitoring device is acquired, and in step S26, monitoring information including the position of the mobile monitoring device and an image at that position is transmitted to a server via wireless or wired communication means. In step S27, it is determined whether or not the end of the rail has been reached, or whether or not the remaining power of the battery mounted on the mobile monitoring device is insufficient. If the determination conditions are met, the process proceeds to step S29-2, where the device returns to the base. If the determination conditions are not met, the process proceeds to step S28.

[0097] In step S28, it is determined whether an abnormal event (monitoring event) has occurred in the image. That is, in the stored image, the AI ​​model is used to identify a specific part in the image (in the case of a chicken, the comb, wattles, etc.), and the warning unit uses the AI ​​model to determine whether the identified specific part is abnormal, and if it is abnormal, it outputs (transmits) warning information including location information of the individual with the specific part and an image including the specific part.

[0098] If an abnormal event has not occurred, the process returns to step S25 and continues moving and capturing images. If an abnormal event has occurred, an alarm may be issued as is, but in order to reduce false recognition, it is preferable to proceed to step S28-1, stop or slow down the mobile monitoring device, and continue observing the image of the abnormal event. Then, if it is determined in step S28-2 that the occurrence of the abnormal event is continuing, an alarm including the position where the abnormal event occurred and the image (including video) at that time is output in step S29. If it is determined in step S28-2 that the occurrence of the abnormal event is not continuing, it is determined that the occurrence is a false recognition, and no alarm is issued, but the process returns to step S25 and movement (monitoring) is continued. After issuing the alarm in step S29, the process proceeds to step S29-1, and, as in step S27, it is determined whether the rail (route) has ended or not, or whether the remaining power of the onboard battery is insufficient or not. If the determination condition is met, the process returns to the base (charging spot) of the mobile monitoring device. If the determination condition is not met, the process returns to step S25 and movement (monitoring) is continued.

[0099] In this example, the determination is made based on an image of a specific part, but the determination may also be made based on the movement of the individual. In that case, the following configuration is used.

[0100] In addition, the monitoring device (livestock barn monitoring device) An image acquisition unit that acquires a video of an individual including location information of the location where the video was taken; A storage unit for storing the acquired video; A generation unit that labels the behavior of each individual in the stored video and generates a trained AI model using an AI technique (e.g., a technique such as reinforcement learning or deep learning); an identification unit that identifies the behavior of an individual in the stored video using an AI model; a warning unit that uses an AI model to determine whether the behavior of the identified individual is abnormal, and if the behavior is abnormal, outputs warning information including location information of the individual and an image including the individual; The present invention is characterized in that it includes: Preferably, among the stored images, images from a specified period (preferably about the past 1-3 weeks) are weighted to generate an AI model. Since the shape, size, color, etc. of each part of livestock change as they grow, this configuration makes it possible to reflect the standard condition of livestock in their current state, making it possible to more appropriately identify abnormal livestock. In addition, by acquiring growth information of livestock in the barn (e.g., 1 week after birth, 2 months after birth, etc.) and customizing the AI ​​model based on this, it becomes possible to more accurately identify, specify, and monitor sick, weakened, and dead livestock.

[0101] Although the present invention has been described in terms of capturing an image of an individual, it is also possible to identify and pinpoint abnormalities in an individual using not only images or videos, but also other measurement information that reflects the condition of the individual or the livestock facility.

[0102] For example, the monitoring device (livestock barn monitoring device) a monitoring information acquisition unit that acquires location information of a monitored location and monitoring information including measurement values ​​(such as sound, odor, temperature, and humidity) measured at the monitored location; A storage unit for storing the acquired monitoring information; A generator that labels the measurements of the stored monitoring information and generates a trained AI model using an AI technique (e.g., a technique such as reinforcement learning or deep learning); an identification unit that identifies measurement values ​​of the stored monitoring information using the AI ​​model; a warning unit that uses the AI ​​model to determine whether the identified measurement value is abnormal, and if the identified measurement value is abnormal, outputs warning information including location information and the measurement information; The present invention is characterized in that it includes:

[0103] FIG. 18 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment (AI model generation) of the present invention. As shown in the figure, an AI model AIM1 (which is numerical information on the color of the comb, preferably specified in RGB) and an AI model AIM2 (which is numerical information on the size of the wattle) are generated from image information SI3 showing chickens BD1-BD7, and stored in a memory unit. There are many methods for generating an AI model, and for example, a specific part, the comb region, may be extracted, color information therein may be extracted, and a label indicating the health condition of each individual may be added. Alternatively, a label indicating the health condition of each individual may simply be added, and the specific parts and their features may be extracted by AI, and an AI model may be generated by having the AI ​​search and learn one specific part related to the health condition (including illness and death), or a combination of two or more specific parts. The next figure shows how to use the AI ​​model. In addition to specific parts, the AI ​​model can also use excrement, environmental temperature, environmental humidity, odor, sound, etc.

[0104] Fig. 19 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (utilizing an AI model). As shown in the figure, the AI ​​model AIM1 is compared with the color of the comb of chicken BD16. In this case, the color of the comb of chicken BD16 falls within the range of the AI ​​model AIM1, so it is determined to match, and chicken BD16 is determined to be normal. Then, another individual is compared.

[0105] On the other hand, the AI ​​model AIM1 is compared with the color of the comb of chicken BD19. In this case, the color of the comb of chicken BD19 is outside the range of the AI ​​model AIM1, so it is determined that there is no match, i.e., that there is an abnormality, and warning information including the position and image of individual BD19 is output.

[0106] FIG. 20 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (application of AI model generation). This adds a label to each individual as to whether the individual is healthy or not (for example, it is preferable to attach only labels of unhealthy, dead, and sick. If there is no label, it can be automatically determined that it is healthy), and learns this to automatically generate an AI model. It is also possible to automatically extract specific information of specific parts that indicate characteristics of abnormality, unhealthy, dead, and sickness using a known AI algorithm. For example, AI model AIM5 is the pattern of specific part 1, AI model AIM6 is the shape of specific part 2, and AI model AIM7 is the temperature of specific part 3, etc. For example, the temperature of a chicken's comb and wattle indicates the health state, and the AIM model AIM7 is automatically generated even if the operator does not specify it. The advantage of this AI is that by simply attaching a label to a certain event (here, the unhealthy state of livestock), the AI ​​extracts features that indicate symptoms of specific parts that would normally be overlooked (temperature, shape, color, pattern, etc.), and an AI model is generated using them. As a result, even in the case of an unknown disease, the AI ​​model can identify features that indicate symptoms of specific parts (temperature, shape, color, pattern, etc.), which can be used to automatically identify and identify unhealthy livestock. Also, while it is possible to generate AI models for individual specific parts or functions, such as AI models AIM5 and AIM6, it is also possible to generate a single AI model that includes all specific parts and functions. With an AI model that includes everything, it is possible to estimate and identify health and abnormal conditions in more detail based on the correlation between the features of each specific part.

[0107] Although the present invention has been described based on the drawings and examples, it should be noted that a person skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present invention. For example, the processes and functions included in each part and each step can be rearranged so as not to be logically contradictory, and multiple means / parts and steps can be combined into one or divided. Alternatively, it should be noted that some components, functions, processes, steps, etc. of the device, method, program, etc. according to the present invention can be arranged on a remote server, etc. In addition, although the present invention has been described as a real-time monitoring device, it can also be used as an editing device or editing system that uses a captured image as an input and extracts the time period or moment when a suspicious object was captured. [Explanation of symbols]

[0108] ACQ……Acquisition Department AIM...Model BD11,BD12...Chicken BD13,BD14...Chicken BD4...Chicken BLD...Building BLD1...Building BR1, BR2... Chicken Coop Striker Stick CAD... Dimensions CG31...Chicken coop CG32...Chicken coop CG32...Chicken coop number CG42...Chicken coop number CI...Information CIALL……Information CM1....Camera CM1: Imaging unit CM2...Camera CM3....Camera CM4...Camera COM……Communication Department CON……Control unit DB...Drone base DIS……Display section DR...Drive unit FL……floor FR1... Flight route FR1,FR2……Flight route FR2... Flight route GD……Guide section GEN……Generation section GL...Guidelines GR...guide rail GR...Unit HDT4...Crest IDP...Identification Department IM...Image information IN... Input section LILI……Position index placement information LSN: Line reading sensor MEM……Memory section MI……Monitoring information MM1...Mobile monitoring device MM2……Mobile monitoring device MM3...Mobile surveillance device MM4...Mobile surveillance device MM5...Mobile surveillance device MM6……Mobile monitoring device MON……Monitoring section MOS1.....Monitoring device NET...Network OUT...Output section PL1……Rotating light RET……Acquisition part RT1...Travel route SBJ1, SBJ2... Target of monitoring SBJ1, SBJ2...Shelf SBJ3, SBJ4...Subject to monitoring SL...Ceiling SL1... Screen interface SL2... Screen interface SL3... Screen interface SLI...Layout information SNSR...sensor SPI……Standard specific part information STR……Storage section SV1...Server SV2...Server WAN……Warning section WRN1...Warning information WRN2...Warning information

Claims

1. A monitoring device, comprising: An image acquisition unit that acquires an image including location information of the location where the image was taken; a storage unit for storing the acquired image and standard specific part information, which is standard physical information of a specific part of an individual; an identification unit for identifying a specific portion within the stored image; a warning unit which compares the identified specific part with standard specific part information stored in the storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, outputs warning information including location information of the individual with the specific part and an image including the specific part; Including, The specific site is a chicken's comb or wattle, The standard specific portion information is standard dimensional information or shape information of the specific portion, the standard specific part information is generated based on images of the individual for a predetermined period stored in the storage unit, The predetermined period is a period during which changes in the specific part associated with the growth of each of the individuals can be understood from the image. A monitoring device comprising:

2. A monitoring device, comprising: an image acquisition unit that acquires an image including location information of the location where the image was taken; a storage unit for storing the acquired image and abnormal specific part information, which is physical information of a specific part of an abnormal individual; an identification unit for identifying a specific portion within the stored image; a warning unit which compares the identified specific part with abnormal specific part information stored in the storage unit, and if the specific part and the abnormal specific part information are within a predetermined numerical value, or if the specific part and the abnormal specific part information match, outputs warning information including position information of the location of the individual in the specific part and an image including the specific part; Including, The specific site is a chicken's comb or wattle, the abnormal specific part information is standard dimensional information or shape information of the specific part, the abnormality specific portion information is generated based on images of the individual for a predetermined period stored in the storage unit, The predetermined period is a period during which changes in the specific part associated with the growth of each of the individuals can be understood from the image. A monitoring device comprising:

3. A monitoring device, comprising: an image acquisition unit that acquires an image of an individual, the image including location information of the location where the image was taken; A storage unit for storing the acquired image; A generating unit that generates standard specific part information, which is standard physical information of at least one specific part of an individual, based on each individual in the stored images; an identification unit for identifying a specific portion within the stored image; a warning unit which compares the identified specific part with the generated standard specific part information, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information differ, outputs warning information including location information of the individual with the specific part and an image including the specific part; Including, The specific site is a chicken's comb or wattle, The standard specific portion information is standard dimensional information or shape information of the specific portion, the standard specific part information is generated based on images of the individual for a predetermined period stored in the storage unit, The predetermined period is a period during which changes in the specific part associated with the growth of each of the individuals can be understood from the image. A monitoring device comprising:

4. 4. The monitoring device according to claim 3, The generation unit, generating standard specific part information, which is standard physical information of at least one specific part of each individual, based on only images of each individual within a predetermined period of time among the stored images; A monitoring device comprising:

5. 5. The monitoring device according to claim 3, The generating unit, the identifying unit, or the warning unit, It operates based on AI techniques, A monitoring device comprising:

6. A monitoring program that causes one or more processors to function as the monitoring device according to any one of claims 1 to 5.

7. A computer-readable storage medium storing the monitoring program according to claim 6.

8. 1. A monitoring method comprising: An image acquisition step in which a computing device acquires an image including location information of the location where the image was taken; a storage step in which the computing device stores the acquired image and standard specific site information, which is standard physical information of a specific site of an individual; an identification step in which the computing device identifies a specific portion in the stored image; a warning step in which the computing device compares the identified specific part with standard specific part information stored in a storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information are different, outputs warning information including position information of the individual with the specific part and an image including the specific part; Including, The specific site is a chicken's comb or wattle, The standard specific portion information is standard dimensional information or shape information of the specific portion, the standard specific part information is generated based on images of the individual for a predetermined period stored in the storage unit, The predetermined period is a period during which changes in the specific part associated with the growth of each of the individuals can be understood from the image. A monitoring method comprising:

9. 1. A monitoring method comprising: An image acquisition step in which a computing device acquires an image including location information of a place where the image was taken; a storage step in which the computing device stores the acquired image and abnormal specific part information, which is physical information of a specific part of an abnormal individual; an identification step in which the computing device identifies a specific portion in the stored image; a warning step in which the computing device compares the identified specific part with abnormal specific part information stored in a storage unit, and if the specific part and the abnormal specific part information are within a predetermined numerical value, or if the specific part and the abnormal specific part information match, outputs warning information including position information of the individual of the specific part and an image including the specific part; Including, The specific site is a chicken's comb or wattle, the abnormal specific part information is standard dimensional information or shape information of the specific part, the abnormality specific portion information is generated based on images of the individual for a predetermined period stored in the storage unit, The predetermined period is a period during which changes in the specific part associated with the growth of each of the individuals can be understood from the image. A monitoring method comprising:

10. 1. A monitoring method comprising: An image acquisition step in which a computing device acquires an image of an individual, the image including location information of the location where the image was taken; A storage step in which the computing device stores the acquired image; A generating step in which the computing device generates standard specific part information, which is standard physical information of at least one specific part of the individual, based on each individual of the stored images; an identification step in which the computing device identifies a specific portion in the stored image; a warning step in which the computing device compares the identified specific part with the generated standard specific part information, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information are different, outputs warning information including position information of the individual of the specific part and an image including the specific part; Including, The specific site is a chicken's comb or wattle, The standard specific portion information is standard dimensional information or shape information of the specific portion, the standard specific part information is generated based on images of the individual for a predetermined period stored in a storage unit; The predetermined period is a period during which changes in the specific part associated with the growth of each of the individuals can be understood from the image. A monitoring method comprising:

11. The monitoring method according to claim 10, The generating step comprises: generating standard specific part information, which is standard physical information of at least one specific part of each individual, based on only images of each individual within a predetermined period of time among the stored images; A monitoring method comprising:

12. The monitoring method according to claim 10 or 11, The generating step, the identifying step, or the warning step, It operates based on AI techniques, A monitoring method comprising:

13. A monitoring program that causes one or more processors to function as the monitoring device according to any one of claims 8 to 12.

14. A computer-readable storage medium storing the monitoring program according to claim 13.

Images