Running detection system, running detection method, and running detection program

The travel detection system uses skeletal information to accurately calculate speed and enforce regulations by analyzing joint points and connections, addressing the challenges of background and size changes, thereby enhancing safety in crowded areas.

WO2024232157A1PCT designated stage expired Publication Date: 2024-11-14KONICA MINOLTA INC

Patent Information

Application Number
PCT/JP2024/009276
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-09
Filing Date
2024-03-11
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Conventional travel detection systems face challenges in accurately calculating the moving speed of individuals due to changes in background, camera location, and varying object sizes, making it difficult to enforce speed regulations in crowded areas.

Method used

A travel detection system that utilizes skeletal information from video acquisition, including joint points and connections, to calculate speed by determining entry and exit points between non-overlapping detection areas, correcting for size changes, and estimating posture to determine compliance with speed regulations.

Benefits of technology

Enables accurate speed calculation and regulation compliance determination regardless of background changes or object size variations, effectively preventing collisions and enforcing speed limits in dynamic environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024009276_14112024_PF_FP_ABST
    Figure JP2024009276_14112024_PF_FP_ABST
Patent Text Reader

Abstract

[Problem] To provide a running detection system, a running detection method, and a running detection program with which it is possible to accurately calculate the moving speed of a person even when the monitoring area, the installed location of a camera, or the like is varied, or when the size of the person changes. [Solution] This running detection system comprises a video acquisition unit 330, a skeleton detection unit 410, and a speed calculation unit 340. The video acquisition unit 330 acquires a video capturing a predetermined area including a moving body. The skeleton detection unit 410 detects skeleton information of the moving body included in the video. The speed calculation unit 340 calculates information about the moving speed of the moving body on the basis of the skeleton information of the moving body.
Need to check novelty before this filing date? Find Prior Art

Description

Travel detection system, travel detection method, and travel detection program

[0001] The present invention relates to a motion detection system, a motion detection method, and a motion detection program.

[0002] Generally, in places where multiple people come and go, whether indoors or outdoors, routes and traffic rules are often specified to avoid collisions and trouble between people. This is because if you suddenly start running or ignore the designated routes in a crowded place, the risk of collisions with others or other accidents increases.

[0003] Such routes and rules of passage are also defined and enforced in workplaces such as corporate offices and factories. Compliance with routes and rules of passage reduces accidents in the work environment, ultimately leading to the realization of a safe and secure society.

[0004] Conventionally, a monitoring system has been known that detects abnormal behavior related to a person's movement based on captured images of the person and issues a notification when an abnormality is detected (Patent Document 1). This monitoring system detects people (passengers boarding or alighting) on ​​a station platform from the size and shape of the person detected using a background subtraction method, and calculates the person's movement vector. The monitoring system then calculates the person's speed from the movement distance between captured images (frames) based on the movement vector, and determines the behavior of a person moving at a certain speed or above as risky behavior. An example of risky behavior is running to board a train.

[0005] Japanese Patent Application Laid-Open No. 2019-41207

[0006] However, in the above-mentioned surveillance system, the background of the person to be detected may change depending on, for example, the surveillance area or the installation location of the camera. Therefore, it is difficult to accurately calculate the movement speed of the person from the shape of the person detected by the background subtraction method. Furthermore, when part of the person's body is hidden or when the size of the person changes depending on the position or movement of the person, it is also difficult to accurately calculate the movement speed of the person from the size and shape of the person detected by the background subtraction method.

[0007] The present invention has been made to solve the above-mentioned problems. That is, a main object of the present invention is to provide a movement detection system, a movement detection method, and a movement detection program that can accurately calculate the movement speed of a person even when the monitoring area or the installation location of the camera is different or the size of the person changes.

[0008] The above-mentioned problems of the present invention are solved by the following means.

[0009] (1) A traveling detection system having an image acquisition unit that acquires an image of a specified area including a moving body, a skeleton detection unit that detects skeletal information of the moving body included in the image, and a speed calculation unit that calculates information regarding the traveling speed of the moving body based on the skeletal information of the moving body.

[0010] (2) The traveling detection system described in (1) above further includes a detection area setting unit that sets multiple non-overlapping detection areas for detecting the moving object in the image of the specified area, and the speed calculation unit calculates information regarding the moving speed of the moving object based on information regarding the travel time from when the moving object detects its entry into one of the multiple detection areas to when it detects its entry into another detection area, and the distance between the one detection area in which the entry of the moving object is detected and the other detection area.

[0011] (3) The skeletal information includes position information of multiple joint points of the moving body and information regarding the connections between the joint points, and the speed calculation unit determines whether a specific joint point of the moving body is located in the one detection area or the other detection area based on the skeletal information of the moving body, and detects the moving body's entry into the one detection area or the other detection area based on the determination result, in the traveling detection system described in (2) above.

[0012] (4) The speed calculation unit estimates the position and / or size of the moving body based on skeletal information of the moving body, and calculates information regarding the moving speed of the moving body based on the estimated position and / or size of the moving body. This is the moving detection system described in (1) above.

[0013] (5) A traveling detection system as described in (4) above, further comprising an attribute information acquisition unit that acquires attribute information of the moving body based on skeletal information of the moving body, and the speed calculation unit calculates information regarding the traveling speed of the moving body based on the skeletal information and the attribute information of the moving body.

[0014] (6) A traveling detection system as described in (5) above, wherein the moving body is a person, the attribute information is information regarding the generation of the moving body, and the speed calculation unit corrects the size of the moving body based on the attribute information and calculates information regarding the moving speed of the moving body based on the corrected size.

[0015] (7) The traveling detection system described in (1) above further comprises a posture estimation unit that estimates the posture of the moving body based on skeletal information of the moving body, and a transportation means determination unit that determines whether the moving body is using transportation means, wherein the moving body is a person, the posture estimation unit determines whether the moving body is standing or sitting, and the transportation means determination unit determines that the moving body is using transportation means if the posture estimation unit determines that the moving body is sitting.

[0016] (8) The traveling detection system according to (2) above, wherein the plurality of detection areas are set to be arranged in a straight line or a curved line in the image of the specified area.

[0017] (9) The traveling detection system according to (1) above, further comprising a recording unit that records the video acquired by the video acquisition unit together with the date and time of shooting.

[0018] (10) A traveling detection system described in any one of (1) to (9) above, further comprising: a regulation setting unit that sets regulations regarding the movement of the moving body; a regulation compliance determination unit that determines whether the movement speed of the moving body satisfies the regulations based on information about the movement speed calculated by the speed calculation unit; and an alarm unit that notifies when the movement speed of the moving body does not satisfy the regulations as a result of the determination by the regulation compliance determination unit.

[0019] (11) The travel detection system described in (10) above, wherein the regulations regarding travel are regulations regarding an upper limit speed, the regulation compliance determination unit determines whether the travel speed of the moving body satisfies the regulations regarding the upper limit speed, and the notification unit notifies when the travel speed of the moving body does not satisfy the regulations regarding the upper limit speed.

[0020] (12) A traveling detection system as described in (11) above, further comprising a memory unit that stores an upper limit speed for each attribute of the moving body, and an attribute information acquisition unit that acquires attribute information of the moving body based on skeletal information of the moving body, wherein the specification setting unit sets an upper limit speed of the moving body according to the attribute information of the moving body.

[0021] (13) A traveling detection method comprising the steps of: (a) acquiring a video of a predetermined area including a moving object; (b) detecting skeletal information of the moving object included in the video; and (c) calculating information relating to the traveling speed of the moving object based on the skeletal information of the moving object.

[0022] (14) The travel detection method described in (13) above further comprises a step (d) of determining whether the travel speed of the moving body satisfies a regulation based on information regarding the travel speed calculated in the step (c), and notifying if the result of the determination is that the travel speed of the moving body does not satisfy the regulation.

[0023] (15) A driving detection program for causing a computer to execute a process including the steps of: (a) acquiring a video of a predetermined area including a moving object; (b) detecting skeletal information of the moving object included in the video; and (c) calculating information related to the moving speed of the moving object based on the skeletal information of the moving object.

[0024] (16) The travel detection program described in (15) above, wherein the processing further includes a step (d) of determining whether the travel speed of the moving body satisfies a regulation based on information regarding the travel speed calculated in step (c), and notifying if the result of the determination is that the travel speed of the moving body does not satisfy the regulation.

[0025] According to the present invention, information regarding the moving speed of a moving object is calculated based on the skeletal information of the moving object, so that the moving object can be detected regardless of the background of the moving object. Furthermore, it is possible to determine whether the entire moving object has been detected based on the skeletal information. Therefore, the moving speed of a person can be calculated with high accuracy even when the monitoring area or the installation location of the camera is different or the size of the person changes.

[0026] 1 is a diagram illustrating a schematic configuration of an information processing system according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating a schematic configuration of an imaging device shown in FIG. 1. FIG. 3 is a schematic block diagram illustrating a hardware configuration of a first information processing device shown in FIG. 1. FIG. 4 is a flowchart illustrating a processing procedure of a travel detection method by the information processing system according to an embodiment. FIG. 5 is a schematic diagram illustrating a method of acquiring an image of a moving body. FIG. 6 is a schematic diagram illustrating a method of setting a detection area for an image of a predetermined area. FIG. 7 is a schematic diagram illustrating a method of detecting entry of a moving body into a detection area. FIG. 8 is a schematic diagram illustrating a method of calculating information related to the moving speed of a moving body in the X direction (when the moving speed is slow). FIG. 9 is a schematic diagram illustrating a method of calculating information related to the moving speed of a moving body in the X direction (when the moving speed is fast). FIG. 10 is a schematic diagram illustrating a method of calculating information related to the moving speed of a moving body in the Y direction. FIG. 11 is a schematic diagram illustrating a notification that the moving speed of a moving body does not satisfy a predetermined requirement. FIG. 12 is a schematic diagram illustrating recording of a determination result. FIG. 13 is a schematic diagram illustrating possible moving directions of a moving body. 10A and 10B are schematic diagrams illustrating an example of privacy protection for a moving object, and an example of an image when a moving object is approaching an imaging device.

[0027] Hereinafter, a motion detection system, a motion detection method, and a motion detection program according to an embodiment of the present invention will be described with reference to the drawings. Note that in the drawings, identical elements are designated by the same reference numerals, and duplicate explanations will be omitted. Also, the dimensional proportions in the drawings are exaggerated for the sake of explanation, and may differ from the actual proportions.

[0028] (Embodiment) A traveling detection system according to this embodiment calculates the traveling speed of a moving object based on skeletal information detected from a video captured of a predetermined area including the moving object. The skeletal information may include a plurality of joint points of the moving object and information regarding the connections between the joint points. The information regarding the connections between the joint points may be represented, for example, by nodes connecting the joint points. The traveling detection system also determines whether the traveling speed of the moving object satisfies a preset specification, and notifies the user if the specification is not satisfied.

[0029] Furthermore, a mobile object may be, for example, an object whose skeleton can be detected, such as a person or an animal. The movement of a mobile object may be, for example, the mobile object walking or running. Running may include the mobile object running itself and the mobile object running using a means of transportation. In this specification, the means of transportation may be, for example, a bicycle, a motorcycle, a wheelchair, a stroller, a kick scooter, a skateboard, a stand-up scooter, a unicycle, etc.

[0030] Furthermore, the provision may be a predetermined rule, agreement, or regulation regarding the moving speed of the moving object. The provision may include, for example, an upper limit and / or lower limit speed, a moving direction (positive or negative value), and the like on the moving route of the moving object. Specifically, the provision may be a law such as the Road Traffic Act (traffic rules) if the moving route of the moving object is on a public road, for example. The provision may be an internal company rule regarding passage if the moving route of the moving object is within a company premises, for example. The provision may be a rule, terms of use, agreement, or the like if the moving route of the moving object is within a public facility such as a train station, library, or children's center, or within a welfare facility such as a hospital or nursing home, for example.

[0031] 1 is a block diagram illustrating a schematic configuration of an information processing system 100 according to an embodiment. The information processing system 100 includes, for example, an imaging device 200, a first information processing device 300, and a second information processing device 400. The imaging device 200 is connected to the first information processing device 300 via a communication network so as to be able to communicate with each other.

[0032] The first information processing device 300 may be, for example, a personal computer or a mobile terminal such as a smartphone or tablet terminal. The second information processing device 400 may be, for example, a server or a personal computer. Note that some or all of the functions of the second information processing device 400 may be performed by the first information processing device 300 or the imaging device 200. Conversely, some or all of the functions of the first information processing device 300 may be performed by the second information processing device 400 or the imaging device 200. The first information processing device 300 and the second information processing device 400 constitute a travel detection system.

[0033] <Image Capture Device 200> Fig. 2 is a block diagram illustrating a schematic configuration of the image capture device 200 shown in Fig. 1. The image capture device 200 includes a control unit 210, a communication unit 220, and a camera 230, which are interconnected by a bus 201. The image capture device 200 is installed in a location where at least a portion of the movement path of a moving object whose movement speed is to be calculated (hereinafter referred to as a "target moving object") falls within the field of view of the camera 230. For example, if the target moving object is a worker working in a company's factory, the image capture device 200 is installed in a location where at least a portion of the worker's movement path falls within the field of view of the camera 230. Furthermore, if the target moving object is a passenger getting on or off a train at a railway station, the image capture device 200 is installed in a location where the station platform and train entrances and exits fall within the field of view of the camera 230.

[0034] The control unit 210 is configured by, for example, a CPU (Central Processing Unit) and memories such as a RAM (Random Access Memory) and a ROM (Read Only Memory). The control unit 210 controls each unit of the image capture device 200 and performs calculation processing in accordance with a travel detection program.

[0035] The control unit 210 transmits a plurality of captured images (frames) obtained by the camera 230 capturing an image of a predetermined area to the first information processing device 300 via the communication unit 220. The predetermined area is a three-dimensional area that includes the movement path of the target moving object. The control unit 210 can also transmit the captured image and the date and time of the image to the first information processing device 300.

[0036] The communication unit 220 includes an interface circuit for communicating with the first information processing device 300 and the like via a communication network, for example.

[0037] The camera 230 may be, for example, a standard camera used for security cameras or surveillance cameras. Alternatively, the camera 230 may be a wide-angle camera with a wider angle of view than a standard camera. For simplicity of explanation, the following description will be given assuming that the camera 230 is a standard camera.

[0038] The camera 230 may be, for example, a visible light camera, which may capture an image of a predetermined area by receiving visible light reflected by an object within the predetermined area using a complementary metal oxide semiconductor (CMOS) sensor.

[0039] Furthermore, camera 230 may be configured using a near-infrared camera instead of a visible light camera. The infrared camera may capture an image of a predetermined area by irradiating the predetermined area with near-infrared light using an LED (Light Emitting Device) and receiving the near-infrared light reflected by objects within the predetermined area using a CMOS sensor. In this case, the image captured by camera 230 may be a monochrome image in which each pixel has a reflectance of near-infrared light.

[0040] The imaging device 200 can capture a predetermined area as a video consisting of a plurality of temporally consecutive captured images (frames) at a frame rate of, for example, 15 fps to 30 fps (see, for example, FIGS. 8 and 9 described below). The camera 230 may also be configured to use both a near-infrared camera and a visible light camera.

[0041] <First information processing device 300> Fig. 3 is a schematic block diagram illustrating an example of the hardware configuration of the first information processing device 300 shown in Fig. 1. The first information processing device 300 has, for example, a CPU 301, a RAM 302, a ROM 303, an auxiliary storage unit 304, a communication I / F 305, and an operation display unit 306. The components of the first information processing device 300 are connected to each other by an internal bus.

[0042] The auxiliary storage unit 304 may be configured, for example, with a solid state drive (SSD) or a hard disk drive (HDD). The auxiliary storage unit 304 stores, for example, a program such as a driving detection program, video captured of a predetermined area including the moving object, the date and time the video was captured, and information regarding regulations. For example, if the target moving object's travel route is on a public road, the information regarding regulations includes information regarding the legal speed, the direction of travel (e.g., one-way streets), and the time period of travel. For example, if the target moving object's travel route is within a company factory, the information regarding regulations includes the speed limit (upper limit, lower limit, etc.) and traffic rules ("keep on the right" or "keep on the left") set forth in company regulations. For example, if the target moving object's travel route is within a train station or public facility building, the information regarding regulations includes the prohibition of running (running) and traffic rules.

[0043] The communication I / F 305 is an interface circuit (for example, a LAN card) for communicating with other devices such as the imaging device 200 and the second information processing device 400 via a network.

[0044] The operation display unit 306 has an input unit and an output unit. The input unit includes, for example, a touch panel, a keyboard, a mouse, etc. The input unit is used by the user to perform touch operations using the touch panel, input characters using the keyboard or mouse, and perform various settings. The output unit includes a display (display device) and displays images including moving objects captured by the imaging device 200.

[0045] 1 again, the functions of the first information processing device 300 will be described. The first information processing device 300 functions as, for example, a detection area setting unit 310, a regulation setting unit 320, an image acquisition unit 330, a speed calculation unit 340, a regulation compliance determination unit 350, a notification unit 360, a recording unit 370, and a display unit 380.

[0046] The detection area setting unit 310 sets, for example, a plurality of detection areas for an image of a predetermined region captured by the imaging device 200. The plurality of detection areas includes, for example, at least a first detection area (e.g., DA1 in FIG. 7 ) and a second detection area (e.g., DA2 in FIG. 7 ) that does not overlap with the first detection area. The plurality of detection areas are used when the speed calculation unit 340 calculates the moving speed of the moving object. Specific examples of the detection areas and details of the setting method will be described later.

[0047] The regulation setting unit 320 sets regulations regarding the movement of the mobile object. Information regarding the regulations set for the mobile object is stored, for example, in the auxiliary storage unit 304. The user can update the information regarding the regulations as needed through the operation and display unit 306 in response to revisions to laws, regulations, rules, etc. The updated information regarding the regulations is overwritten in the auxiliary storage unit 304. Furthermore, when the rules and regulations regarding movement change depending on conditions such as the time of day or weather during which the mobile object is moving, the regulation setting unit 320 can also automatically change the information regarding the regulations in response to the above conditions.

[0048] The video acquisition unit 330 acquires video captured of a predetermined area including a moving object and the date and time of capture of the video. The video may be, for example, multiple frames of images captured by the imaging device 200 of a predetermined area including a moving object. The video acquisition unit 330 acquires the video by receiving it from the imaging device 200 via the communication I / F 305. Furthermore, if the video captured by the imaging device 200 is pre-stored in the auxiliary storage unit 304 or the like, the video acquisition unit 330 can also acquire the video by reading it from the auxiliary storage unit 304 or the like. The video captured by the imaging device 200 may be stored in an external storage device or the like, and the video acquisition unit 330 may be configured to read the video from the storage device. Alternatively, the video captured by the imaging device 200 may be recorded on a recording medium such as a USB memory, and the video acquisition unit 330 may acquire the video offline by reading the video from the recording medium.

[0049] The speed calculation unit 340 calculates information about the moving speed of the target moving object based on the skeletal information of the target moving object. The skeletal information may include position information of multiple joint points, such as the eyes, nose, neck, shoulders, elbows, wrists, waist, knees, and ankles, which are skeletal feature points of the moving object, and nodes connecting the joint points (see, for example, FIG. 7 ). The position information of the joint points may be coordinates (X coordinate, Y coordinate) of the joint points on the captured image. Details of a method for detecting the skeletal information of a moving object will be described later. Note that in this specification, for convenience of illustration, some joint points may be omitted in the drawings. The target moving object may be one or more.

[0050] Furthermore, the information about the moving speed may be the speed (e.g., per second, per minute, per hour, etc.) obtained by dividing the moving time between detection areas in real space. Furthermore, the information about the moving speed may be the number of pixels that the moving object moves on the image per unit time or per frame. Furthermore, as will be described later, the information about the moving speed may be calculated based on the rate of change of the size of the moving object over time.

[0051] The regulation compliance determination unit 350 determines whether or not a predetermined regulation is satisfied with respect to the moving speed of the target moving object (whether or not the regulation is observed). For example, if the moving object travels in a factory at a speed exceeding an upper limit or travels while ignoring the moving direction or route, the regulation compliance determination unit 350 determines that the regulation regarding the moving speed of the moving object is not satisfied.

[0052] The notification unit 360 notifies the user when the target moving object's speed does not satisfy the regulations based on the determination result of the regulation compliance determination unit 350. The notification unit 360 notifies the user that the target moving object's speed does not satisfy the regulations. For example, when the target moving object travels faster than the speed limit, the notification unit 360 may notify the user by displaying this on a display (display DP shown in FIG. 6 ). Furthermore, when the target moving object travels ignoring the travel direction or route, the notification unit 360 may notify the user by displaying this on a display.

[0053] The recording unit 370 records the video acquired by the video acquisition unit 330 together with the date and time of shooting in the auxiliary storage unit 304. The recording unit 370 can also read out the video recorded in the auxiliary storage unit 304 from the input shooting date and time.

[0054] The display unit 380 displays on a display the image acquired by the image acquisition unit 330. The display unit 380 may also display the joint points of the target moving object and the nodes connecting the joint points together by superimposing them on the target moving object.

[0055] 1 , the configuration and functions of the second information processing device 400 will be described. The hardware configuration of the second information processing device 400 is the same as the hardware configuration of the first information processing device 300, so a detailed description will be omitted. The second information processing device 400 functions as a skeleton detection unit 410, an attribute information acquisition unit 420, a posture estimation unit 430, and a transportation means determination unit 440.

[0056] The skeleton detection unit 410 detects skeleton information for a target moving object included in the video acquired by the video acquisition unit 330. The skeleton detection unit 410 detects the skeleton information using a trained model for detecting a person's joint points from a rectangle including a person (hereinafter also referred to as a "person rectangle"). A person rectangle is an area in an image that contains the person's joint points and nodes. Known examples of such trained models include OpenPose (https: / / arxiv.org / abs / 1812.08008), DeepPose (https: / / arxiv.org / abs / 1312.4659), and ResNet (https: / / arxiv.org / abs / 1512.03385).

[0057] The attribute information acquisition unit 420 acquires attribute information of the moving object based on the skeletal information of the moving object. In this embodiment, if the moving object is a person, the attribute information of the moving object may be information about its generation (e.g., adult or child). The attribute information acquisition unit 420 estimates the generation of the target moving object from the detected skeletal information using a trained model for generation estimation that has been trained in advance using the generation of the moving object and the skeletal information of that generation, and outputs the estimated generation as information about the generation of the target moving object.

[0058] The posture estimation unit 430 estimates the posture of the moving object based on the skeletal information of the moving object. For example, the posture estimation unit 430 determines whether the moving object is standing (standing position) or sitting (sitting position) based on the skeletal information of the moving object.

[0059] For example, posture estimation unit 430 estimates the posture of the target moving object from a predetermined relationship based on the positions, movements, etc. of the joint points of the moving object, and outputs the posture-related information. Alternatively, a trained model for posture estimation may be used as a posture estimation method.

[0060] The transportation means determination unit 440 determines whether the target moving object is using transportation means based on information about the posture of the target moving object. More specifically, when the posture estimation unit 430 determines that the moving object is sitting, the transportation means determination unit 440 determines that the moving object is using transportation means.

[0061] The transportation means determination unit 440 can also estimate what type of transportation the target moving object is using based on information about its posture and its moving speed. For example, if the moving object is sitting and moving at a speed faster than a person's running speed, the transportation means determination unit 440 determines that the moving object is using a transportation means such as a bicycle. The transportation means determination unit 440 may also be configured to determine the transportation means used by the moving object based on the positions and movements of the joint points of the moving object. For example, if the moving object is sitting and the ankle joint points of the moving object are moving in a circular motion, the transportation means determination unit 440 determines that the moving object is using a transportation means such as a bicycle.

[0062] <Processing Procedure of Travel Detection Method> FIG. 4 is a flowchart illustrating a processing procedure of a travel detection method performed by the information processing system 100 according to an embodiment. The processing of the flowchart illustrated in FIG. 4 is realized by the CPU 301 executing a travel detection program. FIG. 5 is a schematic diagram illustrating a method of acquiring an image of a moving object, and FIG. 6 is a schematic diagram illustrating a method of setting a detection area for an image captured of a predetermined area. FIG. 7 is a schematic diagram illustrating a method of detecting entry of a moving object into a detection area. FIGS. 8 and 9 are schematic diagrams illustrating a method of calculating information related to the moving speed of the moving object. The moving speed of the moving object in FIG. 9 is assumed to be faster than the moving speed of the moving object in FIG. 8. FIGS. 10 and 11 are schematic diagrams illustrating a method of calculating information related to the moving speed of the moving object in the Y direction. FIG. 12 is a schematic diagram illustrating a notification that the moving speed of the moving object does not satisfy a predetermined requirement, and FIG. 13 is a schematic diagram illustrating recording of a determination result.

[0063] In the following description, it is assumed that at least one person is photographed as the target moving object by the camera 230. For convenience of illustration, the drawings illustrate a case in which one person is photographed, but multiple people may be photographed as the target moving object.

[0064] As shown in Fig. 5 , the imaging device 200 is installed so as to capture an image of a floor surface FL within a predetermined area and a person 500 moving along a movement path PA on the floor surface FL. More specifically, the installation position of the imaging device 200, the orientation of the camera 230 (optical axis direction of the lens), lens distortion, etc. are adjusted so that the floor surface FL within the predetermined area and the standing person 500 fit within the field of view of the camera 230. For example, the imaging device 200 is installed at a predetermined distance from the movement path PA of the person 500. Furthermore, the imaging device 200 is adjusted so that the orientation of the camera 230 is slightly downward relative to the horizontal direction so that the entire floor surface FL within the predetermined area fits within the field of view of the camera 230.

[0065] 4, after the imaging device 200 is installed, the detection area setting unit 310 sets a plurality of detection areas for an image of a predetermined area (step S101). The detection area setting unit 310 sets a plurality of detection areas for an image of the predetermined area captured by the imaging device 200 based on, for example, an input by a user (operator).

[0066] As shown in FIG. 6 , the detection area setting unit 310 sets a two-dimensional area MA in the image IM of a predetermined area, including a path of movement of a person on the floor. For example, the user inputs the positions (X coordinate, Y coordinate) of the four corners of the two-dimensional area MA through the operation and display unit 306 by touching the corresponding positions or clicking the mouse. The detection area setting unit 310 sets the two-dimensional area MA based on the input positions of the four corners. The detection area setting unit 310 also divides the set two-dimensional area MA into multiple grid-like areas (e.g., A11 to A44). In the figure, the depth direction of the image IM captured by the camera 230 of the imaging device 200 (the optical axis direction of the lens of the camera 230) is defined as the Y direction, and the horizontal direction perpendicular to the depth direction is defined as the X direction. For example, each of the areas A11 to A44 corresponds to a rectangular area in real space with a side length of L [m] × M [m]. For example, on the ground or floor, this corresponds to a rectangular area with sides of L [m] (X direction) × M [m] (Y direction). The detection area setting unit 310 converts the distances L [m] and M [m] in real space into the number of pixels in the image IM. Various known methods can be used to convert the distance in real space into the number of pixels in the image IM. For example, the following method can be used as a simple method. When installing the imaging device 200, the detection area setting unit 310 uses the imaging device 200 to capture an image of an object with known lengths in the X and Y directions, and obtains the number of pixels corresponding to the length of the object. Alternatively, on the floor, the imaging device 200 captures an image of an object, such as a pole, that indicates the four corners of a two-dimensional area MA corresponding to the entire area A11 to A44, and obtains the number of pixels corresponding to the length of the integrated rectangular area. The detection area setting unit 310 captures images of the object and obtains the number of pixels at different positions in the Y direction, and then calculates a formula or table for converting the real space distance into the number of pixels.

[0067] The display unit 380 displays grid-like areas A11 to A44 on the display DP based on the number of pixels converted by the detection area setting unit 310. While the figure illustrates an example in which the two-dimensional area MA is divided into four areas in each of the X and Y directions, the number of divisions in the X and Y directions can be set arbitrarily by the user. For example, the user can input the number of divisions in the X and Y directions via the operation display unit 306.

[0068] The detection area setting unit 310 allows the user to select a detection area from these areas A11 to A44. More specifically, the user selects a detection area by, for example, touching the corresponding area on the touch panel. Alternatively, the user selects a detection area by moving a pointer displayed on the display DP to the area corresponding to the detection area and clicking the mouse.

[0069] FIG. 6 illustrates an example in which the first detection area DA1 (area A22) is selected by the user. For example, the user selects the first detection area DA1 by touching area A22 with a finger FI. In the figure, area A22 selected by the user is indicated by diagonal lines. After selecting the first detection area DA1 (area A22), the user can select the second detection area DA2 (area A32).

[0070] Alternatively, the detection area may be designated by the user inputting the range (X and Y coordinates of the upper left and lower right) of any of the areas A11 to A44.

[0071] Next, the rule setting unit 320 sets rules regarding the movement of the moving object (step S102). For example, the user inputs rules for each of the set detection areas via the operation display unit 306, or selects information regarding the rules stored in the auxiliary storage unit 304.

[0072] Next, the video acquisition unit 330 acquires video of a predetermined area including a moving object (step S103). The imaging device 200, for example, captures a person 500 moving in the predetermined area and transmits the captured video of the person 500 to the video acquisition unit 330. The video acquisition unit 330 receives and acquires the video of the person 500 from the imaging device 200, and transmits the acquired video to the second information processing device 400. The recording unit 370 also records the video together with the date and time the video was captured.

[0073] Next, the skeleton detection unit 410 detects skeleton information of the moving object included in the predetermined area (step S104). The skeleton detection unit 410 detects skeleton information of the person HM from a video including multiple frames of captured images acquired by the video acquisition unit 330, and outputs the information to the speed calculation unit 340 and the display unit 380. As shown in FIG. 7 , the display unit 380 uses the skeleton information to superimpose the joint points of the person 500 and the nodes connecting the joint points onto the person HM in the video IM and displays them on the display DP. In the figure, the "circles" displayed at the positions of the person HM's eyes, nose, neck, shoulders, elbows, wrists, waist, knees, ankles, etc. correspond to joint points, and the black lines connecting these "circles" correspond to nodes.

[0074] The velocity calculation unit 340 determines whether a specific joint point (e.g., an ankle) of the person HM is located within the detection area based on the skeletal information of the person HM. Based on the determination result, the velocity calculation unit 340 detects the person HM in the image IM entering the detection area. More specifically, when a specific joint point of the person HM moves from a position outside the first detection area DA1 to a position within the first detection area DA1, the velocity calculation unit 340 detects the person HM entering the first detection area DA1. Similarly, when a specific joint point of the person HM moves from a position outside the second detection area DA2 to a position within the second detection area DA2, the velocity calculation unit 340 detects the person HM entering the second detection area DA2.

[0075] Furthermore, the display unit 380 highlights (or colors in a first color) the second detection area D2 in which the entry of the person HM has been detected. Furthermore, before the entry of the person HM is detected in the second detection area D2, the display unit 380 thickens the outline of the first detection area D1 in which the entry of the person HM has been detected or colors it in a second color. In this way, by making the display forms of the first detection area D1 and the second detection area D2 different from each other, the user can easily determine the entry of the person HM into the detection area.

[0076] Next, the speed calculation unit 340 identifies the target moving object (step S105). When the video includes multiple people, the speed calculation unit 340 identifies the person for whom the moving speed is to be calculated (hereinafter also referred to as the "target person"). The user specifies the target person by, for example, touching or clicking an image area corresponding to the target person on the display DP via the operation display unit 306. Note that multiple target people can also be specified. Furthermore, if no target person is specified, all people included in the video may be identified as target people. In the example shown in FIG. 7, the video includes only one person HM, so the person HM is identified as the target person.

[0077] The speed calculation unit 340 also tracks how each identified target person moves between frames. For example, the speed calculation unit 340 assigns an ID (Identification) to each target person and retains the assigned ID until monitoring ends. This allows the user to individually monitor whether multiple people appearing in the video meet a certain standard. Furthermore, after detecting that a target person does not meet the standard, the user can easily visually identify the target person, thereby easily understanding the presence of the target person.

[0078] Next, the speed calculation unit 340 calculates information regarding the moving speed of the moving object (step S106). As shown in FIG. 8 , for example, assume that the person HM is walking and moving in the X direction. The speed calculation unit 340 detects the person HM's entry into the second detection area DA2 in frame 1, and then detects the person HM's entry into the third detection area DA3 in frame n. The speed calculation unit 340 calculates information regarding the moving speed of the moving object based on information regarding the movement time from when the person HM's entry into the second detection area DA2 is detected to when the person HM's entry into the third detection area DA3 is detected, and the distance between the second detection area DA2 and the third detection area DA3 where the person HM's entry was detected. The distance between the detection areas may be the distance between the detection areas in real space or the number of pixels between the detection areas in the video. Furthermore, the information regarding the movement time may be the time required for the moving object to move the distance between the detection areas, or the number of frames.

[0079] The distance between the second detection area DA2 and the third detection area DA3 of the person HM (the distance traveled linearly by the person HM between the two detection areas) corresponds to, for example, one side L [m] of a square in the real space of the detection area. Furthermore, the travel time T1 from when the person HM enters the second detection area DA2 to when he or she enters the third detection area D3A corresponds to the time calculated by dividing the number of frames n by the frame rate [fps]. Therefore, the speed calculation unit 340 can calculate the average travel speed V1 = L / T1 [m / s] of the person HM from the second detection area DA2 to the third detection area DA3.

[0080] 9, when the number of frames is m (<n), the travel time T2 from when the person HM enters the second detection area DA2 to when he or she enters the third detection area DA3 corresponds to the time calculated by dividing the number of frames m by the frame rate (fps). Therefore, the speed calculation unit 340 can calculate the average travel speed V2 of the person HM from the second detection area to the third detection area as V2 = L / T2 [m / s]. Since T1 > T2, V1 < V2.

[0081] The movement speed of the person HM in the Y direction can also be calculated in the same manner as the movement speed in the X direction. For example, as shown in FIGS. 10 and 11 , assume that the person HM moves from the back of the two-dimensional area MA to the front (i.e., the -Y direction). The speed calculation unit 340 detects the person HM's entry into the fourth detection area DA4, and then detects the person HM's entry into the fifth detection area DA5. The speed calculation unit 340 calculates information regarding the movement speed of the person HM based on information regarding the movement time from when the person HM's entry into the fourth detection area DA4 is detected to when the person HM's entry into the fifth detection area DA5 is detected, and the distance between the fourth detection area DA4 where the person HM's entry was detected and the fifth detection area DA5. In the example shown in FIGS. 10 and 11 , the distance between the fourth detection area DA4 and the fifth detection area DA5 is M [m] × 2, which is one side of a square in the real space of the detection area. Furthermore, the information relating to the movement time is the time T3 obtained by dividing the number of frames from when the person HM enters the fourth detection area DA4 until when the person HM exits the fourth detection area DA4 and enters the fifth detection area DA5 by the frame rate. Therefore, the speed calculation unit 340 can calculate the average movement speed V3 = 2L / T3 [m / s] of the person HM from the fourth detection area DA4 to the fifth detection area DA5.

[0082] In this way, the skeleton detection unit 410 estimates the position (of the ankles) of the person based on the skeleton information of the person HM. The speed calculation unit 340 calculates information related to the moving speed of the person based on the estimated position (of the ankles) of the person.

[0083] Alternatively, the speed calculation unit 340 can calculate the moving speed of the person HM in the Y direction using the following method. The size of the person HM in the image IM is larger when the person HM is in the foreground ( FIG. 11 ) than when the person HM is in the background ( FIG. 10 ). Therefore, information about the moving speed can also be calculated based on the rate of change over time in the size of the moving object.

[0084] The size of the person HM can be estimated, for example, using a human rectangle. The human rectangles BX1 and BX2 shown in Figures 10 and 11 are areas that contain the joint points and nodes of the person HM in the captured image, and can reflect the position, size, and posture of the person HM. In this way, by using the human rectangles BX1 and BX2 that represent the size of the person HM based on skeletal information, it is also possible to deal with changes in the apparent size of the person HM depending on the position in the Y direction.

[0085] Note that the human rectangles BX1 and BX2 can also be estimated using a trained model of a neural network that has been trained in advance to estimate human rectangles from captured images containing people. Examples of trained models for estimating human rectangles from images include R-CNN, Fast R-CNN, Faster R-CNN (https: / / arxiv.org / abs / 1506.01497), YOLO (https: / / arxiv.org / abs / 1506.02640), and SSD (https: / / arxiv.org / abs / 1512.02325).

[0086] On the other hand, the size of the person HM in the image IM can also change depending on the generation of the person HM (adult or child). For example, the body of an adult is generally larger than that of a child. When the body of an adult is larger than that of a child, if the child and adult are at the same position in the Y direction in the image IM, the adult will be larger than the child. In other words, the area of ​​the adult will be larger than the area of ​​the child.

[0087] Therefore, by examining in advance the correspondence between the size of the person rectangle, the person's attribute information, and the person's position in the Y direction, the position of the target person in the Y direction can be estimated from the size of the target person's rectangle.

[0088] Alternatively, by correcting the size of the person based on attribute information and examining in advance the correspondence between the corrected size and the person's position in the Y direction, the position of the target person in the Y direction can be estimated from the size of the target person's rectangle.

[0089] In this way, the speed calculation section 340 can estimate the position of the person HM in the Y direction based on the skeletal information and attribute information of the person HM. Then, the speed calculation section 340 can calculate information related to the movement speed of the person HM based on the estimated position of the person HM.

[0090] Next, the rule compliance determination unit 350 determines whether the moving object's moving speed satisfies the rule (step S107). For example, assume that a rule regarding the moving route in the section from area A21 to area A24 in FIGS. 10 and 11 specifies that the moving object must move in the order of area A21 → A22 → A23 → A24. However, in the example shown in FIGS. 10 and 11, the person HM moves in the order of area A24 → A23 → A22 → A21. In other words, the person HM is traveling in the opposite direction through the section from area A21 to area A24. In this case, the rule compliance determination unit 350 determines that the moving speed of the person HM does not satisfy the rule. Specifically, the rule compliance determination unit 350 makes this determination when the sign of the moving object's moving speed is negative when the direction of movement from area A21 → A22 → A23 → A24 is positive. On the other hand, when the person HM moves in the order of areas A21 → A22 → A23 → A24, the rule compliance determination unit 350 determines that the movement speed of the person HM satisfies the above rule.

[0091] 9, if the upper limit speed Vu in the X direction is set to, for example, V1<Vu<V2, movement at the movement speed V2 exceeds the upper limit speed Vu. In this case, the regulation compliance determination unit 350 determines that the movement of the person HM does not satisfy the regulation of the upper limit speed.

[0092] Furthermore, by setting multiple detection areas, it is possible to detect not only wrong-way driving and speeding, but also sudden U-turns, etc. A sudden U-turn can be, for example, movement from one detection area to another detection area and then returning to the same detection area again in a short time.

[0093] Furthermore, the regulation compliance determination unit 350 can determine whether the movement of the person HM satisfies the regulation based on the determination result of the means of transportation determination unit 440. For example, assume that the movement route of the person HM is within a pedestrian-only zone. If the means of transportation determination unit 440 determines that the person HM is using a bicycle, the regulation compliance determination unit 350 determines that the movement speed of the person HM does not satisfy the regulation.

[0094] Next, if the moving speed of the moving object satisfies the regulation (step S107: YES), the notification unit 360 ends the process. On the other hand, if the moving speed of the moving object does not satisfy the regulation (step S107: NO), the notification unit 360 notifies the fact that the regulation is not satisfied (step S108). As shown in FIG. 12 , if the movement of the person HM does not satisfy the regulation of the moving route, the notification unit 360 displays a warning, for example, "Reverse driving," on the display DP. This notifies the user that the movement of the person HM does not satisfy the regulation of the moving route.

[0095] Furthermore, if the movement of the person HM does not satisfy the speed limit, the notification unit 360 displays a warning, for example, "Exceeding the speed limit," on the display DP. This notifies the user that the movement of the person HM does not satisfy the speed limit.

[0096] Furthermore, the notification unit 360 can color the detection area and / or skeletal line (joint points and nodes) of a person determined to not satisfy the regulations in a predetermined color (e.g., red). Note that in FIG. 12, the detection area is colored black for convenience of illustration. This makes it easier for the user to identify target moving objects that do not satisfy the regulations.

[0097] Furthermore, in this embodiment, the user can later review the video recorded in the auxiliary storage unit 304 over a predetermined period of time, rather than monitoring the video of the moving object in real time. In this case, simply recording and storing the video makes it difficult to identify, for example, when an accident or other problem occurs, any movement that does not meet the rules and is related to the problem. In this embodiment, as shown in FIG. 13 , the recording unit 370 can record the determination result as an "image" with a timestamp. The recording unit 370 records the video with a timestamp in which the notification unit 360 notifies the determination result by adding a warning indicating that the rules do not meet or by coloring the detection area. This makes it easy to review the video after a problem occurs.

[0098] Furthermore, the notification unit 360 can notify or record the determination result by the regulation compliance determination unit 350 in a system (e.g., a server, a mobile terminal of the administrator, etc.) external to the information processing system 100. The notification unit 360 can cooperate with the external system to notify the surrounding area or the administrator of the determination result, for example, by social media, email, or by activating a warning light / buzzer.

[0099] As described above, in the process of the flowchart shown in FIG. 4, the running detection system sets multiple detection areas for video of a predetermined area and sets movement regulations. Next, the running detection system acquires video of a predetermined area including a moving object and detects skeletal information of the moving object contained in the video. Next, the running detection system identifies the target person and calculates information regarding the moving object's movement speed based on the skeletal information of the moving object. The running detection system then determines whether the person's movement speed satisfies regulations and issues an alert if it does not. The process of steps S101 to S108 shown in the figure can be repeated while the running detection of a moving object is being performed, until the process is terminated, for example, by a user's termination instruction or power-off.

[0100] <Possible Directions of Movement of a Moving Object, Number of Detection Areas> Fig. 14 is a schematic diagram illustrating possible directions of movement of a moving object. The moving object may move in any direction, including forward, backward, left, right, or diagonally. In addition, the number of grid-like areas into which the two-dimensional area MA is divided may be increased. In the example shown in the same figure, the two-dimensional area MA is divided into areas A11 to A88 (64 areas).

[0101] Alternatively, three or more detection areas may be set. The detection area setting unit 310 can set three or more non-overlapping detection areas for an image of a predetermined region. The speed calculation unit 340 calculates information regarding the movement speed of the moving object based on information regarding the movement time from when the moving object enters one detection area to when it enters the other detection area for any two of the detection areas, and the distance between the one detection area in which the entry of the moving object is detected and the other detection area.

[0102] Furthermore, the detection areas may be set so as to be arranged in a straight line or a curved line in the image of a predetermined region, as long as they are within the range captured in the image (i.e., within the field of view of the camera 230). For example, the detection areas may be set so as to be arranged in a straight line or a curved line along a movement path consisting of a curve or a combination of curves and straight lines. By increasing the number of grid-like areas and setting the detection areas more finely, the speed calculation unit 340 can calculate the movement speed of the moving object for a smaller area. Therefore, by setting the detection areas more finely along the movement path of the moving object, the speed calculation unit 340 can accurately calculate the movement speed of the moving object even when the moving object moves while turning or meandering.

[0103] <Setting the Upper Limit Speed ​​Vu> In this embodiment, the user can set the upper limit speed Vu of the target moving object for each two-dimensional area MA or for each detection area. For example, if the two-dimensional area MA is indoors, an upper limit speed Vui for indoors can be set, and if the two-dimensional area MA is outdoors, an upper limit speed Vuo for outdoors can be set. Furthermore, an upper limit speed Vu1 can be set between the first detection area DA1 and the second detection area DA2, and an upper limit speed Vu2 different from the upper limit speed Vu1 can be set between the fourth detection area DA4 and the fifth detection area DA5.

[0104] Alternatively, the upper speed limit for each person's attribute may be stored in advance in the auxiliary storage unit 304 (storage unit), and the specification setting unit 320 may be configured to select one of the upper speed limits based on the attribute information of the target person. For example, the upper speed limit Vua and the upper speed limit Vuc may be stored in advance in the auxiliary storage unit 304 for each of the attributes of adults and children, and the specification setting unit 320 may be configured to select the upper speed limit based on whether the target person is an adult or a child. For example, the upper speed limit Vu may be set to a different value when the target person is an adult than when the target person is a child. This is because, for example, if the upper speed limit is set to the running (running) speed of a child, the walking speed (or brisk walking speed) of an adult may exceed the speed limit. In this case, by setting the upper speed limit Vua for an adult target person to a value greater than the upper speed limit Vuc for a child target person, the walking speed (or brisk walking speed) of an adult may be prevented from exceeding the speed limit. Alternatively, the upper limit speed Vua for adults may be set to α times the upper limit speed Vuc for children (for example, 1.0<α<2.0) so that the upper limit speed Vua for adults is linked to the upper limit speed Vuc for children.

[0105] Furthermore, the upper limit speeds for both walking (running) and traveling by transportation means may be stored in advance in the auxiliary storage unit 304, and the specification setting unit 320 may be configured to select the upper limit speed depending on the posture of the target person or the transportation means. For example, the upper limit speeds Vuw and Vub for walking (running) and traveling by bicycle may be stored in advance in the auxiliary storage unit 304, and the upper limit speed may be selected depending on whether the target person is standing or sitting (using a bicycle). The posture estimation unit 430 may determine that the target person is walking (running) when the posture of the target moving body is standing, and may determine that the target person is traveling by bicycle when the posture of the target moving body is sitting.

[0106] For example, if the two-dimensional area MA includes a sidewalk and a bicycle lane, the upper speed limit Vuw is selected for a target person walking on the sidewalk, while the upper speed limit Vub is selected for a target person riding a bicycle in the bicycle lane.

[0107] <Privacy Protection> FIG. 15 is a schematic diagram illustrating privacy protection for a moving object. From the perspective of protecting the privacy of the target person HM, the display unit 380 can apply a mosaic MZ to body parts of the target person HM that can identify the target person HM (e.g., the face, upper body, etc.). This allows the target person HM to be watched or monitored while protecting their privacy when the video is visible to people other than the user. Note that when displaying video in real time, the display unit 380 can apply a mosaic MZ to the target person HM in the video acquired by the video acquisition unit 330 and display the video. Furthermore, the recording unit 370 can record the video acquired by the video acquisition unit 330 as is, or can record the video with the mosaic MZ applied to the target person HM in the auxiliary storage unit 304.

[0108] The movement detection system of the present embodiment described above can achieve the following effects.

[0109] Since information regarding the moving speed of the moving object is calculated based on the skeletal information of the moving object, the moving object can be detected regardless of the background of the moving object. Furthermore, it is possible to determine whether the entire moving object has been detected based on the skeletal information. Therefore, the moving speed of a person can be calculated with high accuracy even if the monitoring area or the installation location of the camera 230 is different or the size of the person changes.

[0110] Furthermore, in areas where an unspecified number of moving objects pass through, the risk of a moving object falling or colliding can be detected before it actually occurs. Therefore, in areas where restrictions on the number of people allowed in are imposed, the need for restrictions can be detected and the number of people allowed in can be estimated based on the risk of a falling or collision.

[0111] Furthermore, in areas where certain movements such as driving forward, driving in the wrong direction, or making sudden U-turns are prohibited, these specific movements can be detected in real time or from recorded video and reported.

[0112] In addition, in this embodiment, the moving speed of the moving object is calculated rule-based based on the skeleton information detected by the trained model (machine learning) and parameters (information related to the detection area and regulations). Therefore, even if a user is not familiar with machine learning, the moving speed of the moving object can be easily calculated by simply setting and adjusting the parameters.

[0113] Furthermore, in this embodiment, if the entire moving object is not included in the image, the moving object can be excluded from the calculation of the moving speed. FIG. 16 is a schematic diagram illustrating an example of an image in which a moving object is approaching the imaging device 200. For example, if a person 500 approaches the camera 230 and walks (runs), the entire body of the person HM may not fit within the field of view of the camera 230. In the example shown in the figure, the lower half of the person HM's body is not included in the image IM, so the knee and ankle joint points are not detected. Therefore, when detecting the entry of the person HM into the detection area based on the position of the ankle joint point as described above, the speed calculation unit 340 cannot detect the entry of the person HM into the detection area. In this case, the speed calculation unit 340 cannot calculate the moving speed of the person 500. Furthermore, the notification unit 360 can be configured to notify the user that the moving speed cannot be calculated. As a result, in this embodiment, the moving speed of the person 500 is not calculated, thereby preventing erroneous determinations.

[0114] In contrast, when detecting a person as an object and estimating the person's movement speed, as in the past, it is not possible to distinguish whether the image shows the entire person or only half of the body. Therefore, if a person is walking (running) close to the camera, the person's movement speed can be estimated, but the estimated movement speed may be too high. As a result, the person may be mistakenly determined to be running.

[0115] The configuration of the information processing system 100 described above is a main configuration for explaining the features of the above embodiment, but is not limited to the above configuration and can be modified in various ways within the scope of the claims. Furthermore, configurations that are included in general information processing systems are not excluded.

[0116] For example, in the above-described embodiment, the case where the second information processing device 400 has the skeleton detection unit 410, and the first information processing device 300 and the second information processing device 400 function as a running detection system has been exemplified. However, the present invention is not limited to this case, and the first information processing device 300 may be configured to have the skeleton detection unit 410 and function as a running detection system.

[0117] Furthermore, in the above-described embodiment, a case where a user selects multiple detection areas has been described, but the present invention is not limited to such a case. For example, when multiple detection areas are set for an area whose distance in real space is known, or when the movement path of a moving object is fixed, the multiple detection areas may be configured to be set in advance.

[0118] Furthermore, the speed calculation unit 340 can also detect and notify "running" of, for example, both "adults" and "children," or only one of them, based on the attribute information.

[0119] The means and methods for performing various processes in the information processing system 100 described above can be realized by either dedicated hardware circuits or a programmed computer. The programs may be provided, for example, by a computer-readable recording medium such as a USB memory or a DVD (Digital Versatile Disc)-ROM, or may be provided online via a network such as the Internet. In this case, the programs recorded on the computer-readable recording medium are typically transferred and stored in a storage unit such as a hard disk. The programs may also be provided as standalone application software or may be incorporated as a function into the software of a server or other device.

[0120] This application is based on Japanese Patent Application No. 2023-077080, filed on May 9, 2023, the disclosure of which is incorporated by reference in its entirety.

[0121] 100 Information processing system, 200 Imaging device, 201 Bus, 210 Control unit, 220 Communication unit, 230 Camera, 300 First information processing device, 301 CPU, 302 RAM, 303 ROM, 304 Auxiliary storage unit, 305 Communication I / F, 306 Operation display unit, 310 Detection area setting unit, 320 Regulation setting unit, 330 Image acquisition unit, 340 Speed ​​calculation unit, 350 Regulation compliance determination unit, 360 Notification unit, 370 Recording unit, 380 Display unit, 400 Second information processing device, 410 Skeleton detection unit, 420 Attribute information acquisition unit, 430 Posture estimation unit, 440 Means of transportation determination unit.

Claims

1. A moving detection system having an image acquisition unit that acquires an image of a specified area including a moving object, a skeleton detection unit that detects skeleton information of the moving object contained in the image, and a speed calculation unit that calculates information regarding the moving speed of the moving object based on the skeleton information of the moving object.

2. A traveling detection system as described in claim 1, further comprising a detection area setting unit which sets multiple non-overlapping detection areas for detecting the moving object in the image of the specified area, and the speed calculation unit calculates information regarding the moving speed of the moving object based on information regarding the movement time from when the moving object's entry into one of the multiple detection areas is detected to when the moving object's entry into another detection area is detected, and the distance between the one detection area in which the entry of the moving object is detected and the other detection area.

3. The traveling detection system of claim 2, wherein the skeletal information includes position information of a plurality of joint points of the moving body and information regarding the connections between the joint points, and the speed calculation unit determines whether a specific joint point of the moving body is located in the one detection area or the other detection area based on the skeletal information of the moving body, and detects the moving body's entry into the one detection area or the other detection area based on the determination result.

4. A traveling detection system as described in claim 1, wherein the speed calculation unit estimates the position and / or size of the moving body based on skeletal information of the moving body, and calculates information regarding the moving speed of the moving body based on the estimated position and / or size of the moving body.

5. A traveling detection system as described in claim 4, further comprising an attribute information acquisition unit that acquires attribute information of the moving body based on skeletal information of the moving body, and the speed calculation unit calculates information regarding the traveling speed of the moving body based on the skeletal information and the attribute information of the moving body.

6. A traveling detection system as described in claim 5, wherein the moving body is a person, the attribute information is information relating to the generation of the moving body, and the speed calculation unit corrects the size of the moving body based on the attribute information, and calculates information regarding the moving speed of the moving body based on the corrected size.

7. The travel detection system of claim 1, further comprising: a posture estimation unit that estimates the posture of the moving body based on skeletal information of the moving body; and a mobility means determination unit that determines whether the moving body is using a mobility means, wherein the moving body is a person, the posture estimation unit determines whether the moving body is standing or sitting, and the mobility means determination unit determines that the moving body is using a mobility means when the posture estimation unit determines that the moving body is sitting.

8. The motion detection system according to claim 2, wherein the plurality of detection areas are set to be arranged in a straight line or a curved line in the image of the specified region.

9. A motion detection system as claimed in claim 1, further comprising a recording unit which records the image acquired by said image acquisition unit together with the date and time of shooting.

10. A movement detection system as described in any one of claims 1 to 9, further comprising: a regulation setting unit that sets regulations regarding the movement of the moving body; a regulation compliance determination unit that determines whether the moving speed of the moving body satisfies the regulations based on information regarding the moving speed calculated by the speed calculation unit; and a notification unit that notifies when the moving speed of the moving body does not satisfy the regulations as a result of the determination by the regulation compliance determination unit.

11. The travel detection system of claim 10, wherein the regulation regarding movement is a regulation regarding an upper limit speed, the regulation compliance determination unit determines whether the movement speed of the moving body satisfies the regulation regarding the upper limit speed, and the notification unit issues an alert when the movement speed of the moving body does not satisfy the regulation regarding the upper limit speed.

12. A traveling detection system as described in claim 11, further comprising: a memory unit that stores an upper limit speed for each attribute of the moving body; and an attribute information acquisition unit that acquires attribute information of the moving body based on skeletal information of the moving body, wherein the specification setting unit sets the upper limit speed of the moving body according to the attribute information of the moving body.

13. A method of detecting movement, comprising the steps of: (a) acquiring an image of a specified area including a moving object; (b) detecting skeletal information of the moving object contained in the image; and (c) calculating information relating to the movement speed of the moving object based on the skeletal information of the moving object.

14. A travel detection method as described in claim 13, further comprising a step (d) of determining whether or not the travel speed of the moving body satisfies a regulation based on information regarding the travel speed calculated in step (c), and notifying if the result of the determination is that the travel speed of the moving body does not satisfy the regulation.

15. A movement detection program for causing a computer to execute a process including the steps of: (a) acquiring an image of a specified area including a moving object; (b) detecting skeletal information of the moving object contained in the image; and (c) calculating information relating to the movement speed of the moving object based on the skeletal information of the moving object.

16. A movement detection program as described in claim 15, wherein the processing further includes a step (d) of judging whether the movement speed of the moving body satisfies a regulation based on information regarding the movement speed calculated in step (c), and notifying if the result of the judgment shows that the movement speed of the moving body does not satisfy the regulation.

Citation Information

Patent Citations

  • Monitoring system and monitoring method

    JP2019041207A

  • Abnormality detection device

    JP2023077080A

  • Apparatus for detecting situation of platform

    JP2012121483A

  • Movement information processing device and method

    JP2015042241A

  • Photographing system and photographing method

    JP2022057885A

Cited By

  • Image rotation processing device, image rotation processing method and program

    JP7804823B1