Program, information processing system and information processing method

Abstract

A program, an information processing system, and an information processing method for accurately detecting a moving object in a captured image of the moving object are provided. [Solution] A program that causes a computer to function as a training image generation unit 111, a position identification unit 112, and a detection model generation unit 113, wherein the training image generation unit 111 generates a training image that is a two-dimensional training image onto which a moving object model and a background model arranged in a three-dimensional virtual space are projected, the training image including an afterimage corresponding to the movement of the moving object model, the position identification unit 112 identifies the position within the training image of a moving object region that includes an image of at least one moving object model, and the detection model generation unit 113 uses a combination of the training image and the position within the image of the moving object region as training data to generate a detection model that detects a moving object region from a captured image including an afterimage corresponding to the movement of the moving object.

Description

[Technical Field]

[0001] The present invention relates to a program, an information processing system, and an information processing method. [Background technology]

[0002] Conventionally, there is known a technique for generating a detection model for detecting a predetermined object from a captured image by machine learning. Patent Document 1 discloses a technique for generating an image in which a region including the object is cut out using an automatic cutout device generated by learning based on a manually cut-out image in which a region including the object is manually cut out from a material image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-046858 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, in a photographed image of a ball in flight, if the flying speed is high, the image of the ball may appear linear, which is called an afterimage. In photographed images containing such afterimages, it is difficult to accurately identify the moving object.

[0005] The present invention has been made in consideration of the above points, and an object of the present invention is to provide a program, an information processing system, and an information processing method for accurately detecting a moving object in a photographed image of the moving object. [Means for solving the problem]

[0006] The program of the present invention is A program that causes a computer to function as a learning image generation unit, a position identification unit, and a detection model generation unit, the training image generation unit generates a two-dimensional training image onto which a moving body model and a background model arranged in a three-dimensional virtual space are projected, the training image including an afterimage according to the movement of the moving body model; the position specifying unit specifies an intra-image position within the training image of a moving object region including at least one image of the moving object model; The detection model generation unit generates a detection model that detects moving object regions from a captured image that includes afterimages corresponding to the movement of a moving object by using a combination of the training image and the position of the moving object region within the image as training data.

[0007] In the program of the present invention, The training image generation unit may generate a plurality of two-dimensional images in which the dynamic body model is positioned at different positions in accordance with movement of the dynamic body model, and generate the training image based on the plurality of two-dimensional images.

[0008] In the program of the present invention, The position specifying unit may specify, in the training image, a position of an area including the plurality of moving body models as the position within the image of the moving body area.

[0009] In the program of the present invention, The training image generation unit may generate a two-dimensional image including a moving body model and a background model arranged in a three-dimensional virtual space, and generate the training image by performing a blurring process on the image of the moving body model included in the two-dimensional image.

[0010] In the program of the present invention, The position specifying unit may specify a position within the image of the moving object region including the blurred region.

[0011] In the program of the present invention, The position specifying unit may specify, as the intra-image position, a position in the learning image that corresponds to a position of the moving object region in the two-dimensional image before the blurring process is performed.

[0012] In the program of the present invention, The training image generation unit may generate a plurality of two-dimensional images in which the moving body model is positioned at different positions according to the movement of the moving body model, generate a composite image based on the plurality of two-dimensional images, and generate the training image by blurring the image of the moving body model included in the composite image.

[0013] In the program of the present invention, The moving body region may be a rectangular region surrounding at least one image of the moving body model.

[0014] The program of the present invention is A program that causes a computer to function as a photographed image acquisition unit and a moving object region detection unit, the captured image acquisition unit acquires a captured image including an afterimage according to the movement of a moving object; the moving object region detection unit detects the moving object region from the photographed image acquired by the photographed image acquisition unit by utilizing a detection model that detects a moving object region including a moving object from the photographed image including an afterimage corresponding to the movement of the moving object; The detection model is generating a two-dimensional learning image onto which a moving body model and a background model arranged in a three-dimensional virtual space are projected, the learning image including an afterimage according to the movement of the moving body model; Identifying an intra-image position within the training image of a moving object region including at least one image of the moving object model; The image is generated by using a combination of the training image and the position of the moving object region within the image as training data.

[0015] The information processing system of the present invention comprises: a training image generation unit that generates a two-dimensional training image onto which a moving body model and a background model arranged in a three-dimensional virtual space are projected, the training image including an afterimage according to the movement of the moving body model; a position specifying unit that specifies a position within the training image of a moving object region that includes at least one image of the moving object model; a detection model generation unit that generates a detection model for detecting a moving object region from a captured image including an afterimage corresponding to the movement of the moving object by using a combination of the training image and the position of the moving object region in the image as training data; The present invention is characterized by comprising:

[0016] The information processing system of the present invention comprises: a captured image acquisition unit that acquires a captured image including an afterimage according to the movement of a moving object; a moving object region detection unit that detects a moving object region including a moving object from a photographed image that includes an afterimage corresponding to the movement of the moving object, using a detection model for detecting the moving object region from the photographed image acquired by the photographed image acquisition unit; Equipped with The detection model is generating a two-dimensional learning image onto which a moving body model and a background model arranged in a three-dimensional virtual space are projected, the learning image including an afterimage according to the movement of the moving body model; Identifying an intra-image position within the training image of a moving object region including at least one image of the moving object model; The image is generated by using a combination of the training image and the position of the moving object region within the image as training data.

[0017] The information processing method of the present invention comprises: An information processing method performed by a computer having a control unit, a step of generating, by the control unit, a two-dimensional learning image onto which a moving body model and a background model arranged in a three-dimensional virtual space are projected, the learning image including an afterimage according to the movement of the moving body model; a step of the control unit specifying an intra-image position within the training image of a moving object region including at least one image of the moving object model; a step in which the control unit generates a detection model for detecting a moving object region from a captured image including an afterimage corresponding to movement of the moving object by using a combination of the learning image and the position in the image of the moving object region as learning data; The present invention is characterized in that it includes:

[0018] The information processing method of the present invention comprises: An information processing method performed by a computer having a control unit, a step of the control unit acquiring a photographed image including an afterimage according to the movement of a moving object; a step in which the control unit detects a moving object region from the captured image acquired by the captured image acquisition unit using a detection model that detects a moving object region including a moving object from the captured image including an afterimage corresponding to the movement of the moving object; Including, The detection model is generating a two-dimensional learning image onto which a moving body model and a background model arranged in a three-dimensional virtual space are projected, the learning image including an afterimage according to the movement of the moving body model; Identifying an intra-image position within the training image of a moving object region including at least one image of the moving object model; The image is generated by using a combination of the training image and the position of the moving object region within the image as training data. [Effects of the Invention]

[0019] According to the program, information processing system, and information processing method of the present invention, it is possible to provide a technique for accurately detecting a moving object in a captured image that includes the moving object. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is an overall configuration diagram of an information processing system. [Figure 2] FIG. 10 is a diagram illustrating the flow of data in a learning unit and an estimation unit. [Figure 3] FIG. 10 is an explanatory diagram of a learning image. [Figure 4]FIG. 10 is an explanatory diagram of a moving object region. [Figure 5] 10 is a flowchart showing a learning process. [Figure 6] 10 is a flowchart showing a detection process. [Figure 7] FIG. 10 is an explanatory diagram of a moving object region according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0021] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will now be described with reference to the accompanying drawings, in which: Figures 1 to 7 are diagrams showing a program, an information processing system, an information processing method, and the like according to the present embodiment.

[0022] 1 is a diagram showing the overall configuration of an information processing system 1 according to this embodiment. The information processing system 1 detects a moving object region, which is a region of an image of a moving object, from a captured image captured by a camera 210 and including an afterimage corresponding to the movement of the moving object.

[0023] Examples of moving objects to be detected include a baseball thrown by a pitcher, a tennis ball thrown by a tennis racket, a golf ball thrown by a golf club, a ball released from a device such as a pitching machine, etc. Note that the moving object to be detected is not limited to a ball, and may be any moving object that moves too fast in a captured image compared to the shutter speed, causing the object to appear linearly along its line of movement, i.e., leaving a so-called afterimage.

[0024] The information processing system 1 includes a server device 10 and a mobile terminal 20. The server device 10 and the mobile terminal 20 are connected to each other so as to be able to communicate with each other via a communication network 2 such as the Internet. Here, the server device 10 is configured by a computer or the like, and includes a control unit 100, a communication unit 140, a storage unit 150, a display unit 160, and an operation unit 170.

[0025] The control unit 100 includes a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and controls the operation of the server device 10. The communication unit 140 includes a communication interface that communicates with an external device wirelessly or via communication. The control unit 100 transmits and receives data to and from the mobile terminal 20 via the communication unit 140.

[0026] The storage unit 150 includes, for example, a hard disk drive (HDD), a random access memory (RAM), a read only memory (ROM), and a solid state drive (SSD). The storage unit 150 is not limited to being built into the server device 10, but may be a storage medium (for example, a USB memory) that is detachably attached to the server device 10. In this embodiment, the storage unit 150 stores programs executed by the control unit 100 and detection models. The detection models will be described later.

[0027] The display unit 160 is, for example, a monitor, and is configured to display various screens by receiving display commands from the control unit 100. The operation unit 170 is, for example, a keyboard, and can give various commands to the control unit 100.

[0028] The mobile terminal 20 includes a communication unit 200, a camera 210, and a display unit 220. The communication unit 200 includes a communication interface for communicating with an external device wirelessly or via a wired connection. The camera 210 captures an image. The image captured by the camera 210 is transmitted to the server device 10 via the communication unit 200. The display unit 220 is, for example, a monitor, and displays various screens. The display unit 220 displays, for example, the captured image.

[0029] For example, a smartphone, a PC tablet, etc. may be used as the mobile terminal 20. Furthermore, a camera or the like capable of communicating with an external device may also be used as the mobile terminal 20.

[0030] In this embodiment, a photography application can be installed on mobile terminal 20. When the installed photography application is started on mobile terminal 20, the photography application enables camera 210 to take a photographic image.

[0031] Furthermore, information on the shooting date and time of the captured image is acquired by the photography application of the mobile terminal 20. Then, when the captured image is acquired, the captured image and the shooting date and time are transmitted from the mobile terminal 20 to the server device 10 in association with each other.

[0032] Next, a description will be given of the configuration of the control unit 100 of the server device 10. The control unit 100 functions as a learning unit 110, a detection unit 120, and a communication processing unit 130 by executing a program stored in a storage unit 150, which will be described later.

[0033] The learning unit 110 is a functional part that learns a detection model used to detect a moving object from a captured image. The detection model of this embodiment detects a moving object from a captured image that includes an afterimage of the moving object. More specifically, the detection model detects a moving object region within a captured image that includes an afterimage corresponding to the movement of the moving object. The moving object region is the smallest rectangular region that includes an image of the moving object to be detected, a so-called bounding box. As another example, the moving object region may be a region whose boundary is the boundary line of the moving object to be detected.

[0034] The detection unit 120 is a functional part that acquires an image captured by the camera 210 in the mobile terminal 20 and detects a moving object region in the captured image. In detecting the moving object region, the detection model generated in the learning unit 110 is used.

[0035] The learning unit 110 has a training image generation unit 111, a position identification unit 112, and a detection model generation unit 113. The detection unit 120 has a photographed image acquisition unit 121 and a moving object region detection unit 122. Note that, below, the processes described as being performed by the training image generation unit 111, the position identification unit 112, the detection model generation unit 113, the photographed image acquisition unit 121, the moving object region detection unit 122, and the communication processing unit 130 are processes performed by the control unit 100 executing a program.

[0036] FIG. 2 is a diagram illustrating the flow of data in the learning unit 110 and the detection unit 120. The learning unit 110 generates learning data used to learn a detection model. Here, the learning data is data combining learning images and positions within the images. Here, the learning images included in the learning data are two-dimensional images generated for learning that include afterimages of moving objects. The positions within the images included in the learning data are information indicating the positions within the learning images of moving object regions detected from the learning images. Note that the positions within the images of moving object regions are information indicating where the moving object regions are located within the images, and more specifically, are information indicating the boundaries of the moving object regions. The information indicating the boundaries of the moving object regions may be, for example, coordinate information indicating the boundaries of the moving object regions, or may be so-called mask image information that specifies whether each pixel is included in the moving object region.

[0037] The training image generation unit 111 generates training images. The process of generating training images will be described below. The training image generation unit 111 first arranges a moving body model represented by three-dimensional computer graphics (3DCG) and a background in a virtual three-dimensional space. Here, the moving body model is formed by at least one polygon. The training image generation unit 111 arranges a virtual camera in the virtual space in which the multiple moving body models and the background are arranged, and generates a two-dimensional image by virtually capturing an image of the virtual space with the virtual camera.

[0038] As a result, for example, a two-dimensional image 301 shown in Fig. 3 is generated. Two-dimensional image 301 includes a moving object model 311 projected in two dimensions and a background 312. Two-dimensional image 301 shown in Fig. 3 is an image that includes an afterimage corresponding to the flight of a baseball thrown by a pitcher. In other words, moving object model 311 is a 3D model representing a baseball.

[0039] Furthermore, the training image generation unit 111 generates a plurality of two-dimensional images along the movement of the moving object. As a result, for example, three two-dimensional images 301 to 303 along the movement of the moving object are generated, as shown in Fig. 3. Here, it is assumed that a baseball as a moving object moves from left to right in the two-dimensional image, and correspondingly, in two-dimensional images 301 to 303, moving object model 311 moves sequentially from left to right. In other words, the position of moving object model 311 differs in two-dimensional images 301 to 303.

[0040] The training image generation unit 111 then averages the pixel values ​​of the generated multiple 2D images to obtain one training image. As a result, as shown on the right side of FIG. 3, a training image 320 is generated, which includes a series of afterimage images 321 of multiple moving objects, representing the afterimage of the moving object. In this manner, the training image generation unit 111 generates multiple 2D images and then averages the pixel values ​​of the multiple 2D images to generate a training image including an afterimage. Note that the multiple 2D images generated by the training image generation unit 111 are multiple 2D images that correspond to the movement of moving objects at regular time intervals. However, the time intervals between the multiple 2D images do not have to be constant. For example, the time interval between earlier 2D images may be wider than the time interval between later 2D images. This generates a training image that is closer to a frame obtained at a later timing.

[0041] For ease of explanation, Fig. 3 shows an example in which a training image is generated from three two-dimensional images. However, in order to obtain a training image that is closer to an actual captured image including an afterimage, it is preferable to generate more two-dimensional images and average their pixel values ​​to generate a training image. In generating the training image, information that reproduces the material, etc., of each surface of the moving body model and background may be given, and the intensity of the light irradiating the moving body model, the position of the light source, etc. may be set.

[0042] The position identification unit 112 shown in FIG. 2 identifies the intra-image position of the moving object region in the training image generated by the training image generation unit 111. In this way, the position identification unit 112 annotates the intra-image position of the moving object region. In this embodiment, the position identification unit 112 identifies the position of a rectangular region surrounding a moving object model in the 2D image corresponding to the latest time in chronological order among the multiple 2D images used to generate the training image as the intra-image position in the training image. Note that the positions in the 2D image and the positions in the training image are represented by the same coordinate system. Furthermore, the position of the moving object model in the captured image is obtained by projecting and transforming the placement position of the moving object model in virtual space. At this time, the position of the virtual camera, etc. are referenced.

[0043] As shown in Fig. 3, when it is assumed that a baseball is flying from left to right in the two-dimensional image, the area surrounding the rightmost moving object model 311 is identified as moving object area 330 as shown in Fig. 4. In this way, annotation can be performed automatically. As another example, annotation may be performed in response to a user operation.

[0044] The detection model generation unit 113 shown in FIG. 2 acquires, as training data, a combination of the training image obtained by the training image generation unit 111 and the position in the image of the moving object region obtained from the training image. The training image generation unit 111 generates a plurality of different training images, and the position identification unit 112 acquires a plurality of pieces of training data corresponding to each of the plurality of different training images. The position identification unit 112 then generates a detection model 151 by machine learning using the plurality of pieces of training data. Various known methods such as deep learning can be used as the machine learning. The detection model 151 is stored in the storage unit 150.

[0045] In the detection unit 120, the captured image acquisition section 121 acquires the captured image captured by the camera 210 in the mobile terminal 20 via the communication section 140. It is assumed that the captured image includes an afterimage of a moving object.

[0046] The moving object region detection unit 122 detects a moving object region from a captured image using the detection model 151 generated by the learning unit 110 and stored in the storage unit 150. As a result, even if the captured image contains an afterimage, it is possible to detect the region of the afterimage in which a moving object exists at the latest timing as the moving object region. Specifically, the moving object region detection unit 122 identifies information indicating the boundary of the moving object region.

[0047] Next, the processing by the learning unit 110 will be described. FIG. 5 is a flowchart showing the learning processing by the learning unit 110. In the learning processing, first, the learning image generation unit 111 of the learning unit 110 generates, as learning data, a learning image including an afterimage of a moving object using 3DCG (step S100). Next, the position identification unit 112 identifies the intra-image position of the moving object region in the learning image (step S102). Through the above processing, learning data is generated, which is a combination of the learning shadow image and the intra-image position of the moving object region in the learning image. Next, the detection model generation unit 113 generates a detection model for detecting a moving object from a captured image including an afterimage based on multiple pieces of learning data (step S104). The detection model is stored in the storage unit 150.

[0048] Next, the processing by the detection unit 120 will be described. Fig. 6 is a flowchart showing the detection processing by the detection unit 120. In the detection processing, the captured image acquisition unit 121 of the detection unit 120 acquires a captured image including an afterimage, captured by the camera 210 (step S200). Next, the moving object region detection unit 122 detects a moving object region from the captured image using a detection model stored in the storage unit 150 (step S202).

[0049] As described above, according to the information processing system 1 of this embodiment, it is possible to accurately detect a moving object region even in a captured image that includes an afterimage. Furthermore, according to the information processing system 1 of this embodiment, it is possible to generate a detection model for accurately detecting a moving object region from a captured image that includes an afterimage. Furthermore, in generating the detection model, it is possible to generate training images using 3DCG, so it is possible to efficiently collect training data.

[0050] The program, computer, information processing system, information processing method, etc. of the present invention are not limited to the above-described embodiments, and various modifications can be made.

[0051] As a first modification, as shown in Fig. 7, in the detection model learning process, the position identifying unit 112 may identify a region 340 in the learning image that includes the entire afterimage of the moving object as a moving object region, and identify the position of the moving object region within the image. Here, the region 340 that includes the entire afterimage of the moving object is a region that includes multiple moving object models included in each of the multiple two-dimensional images used to generate the learning image. For example, in the example of three two-dimensional images 301 to 303 in Fig. 3, a region that includes all of the region of the moving object model 311 in the two-dimensional image 301, the region of the moving object model 311 in the two-dimensional image 302, and the region of the moving object model 311 in the two-dimensional image 303 is identified as the moving object region.

[0052] The position identifying unit 112 may identify, as the moving object region, a region that includes at least one moving object model from among the region 340 that includes the entire afterimage of the moving object. As yet another example, the position identifying unit 112 may identify, as the moving object region, a region that corresponds to the moving object captured at the earliest timing from among the regions that include the afterimage of the moving object. In this case, the position identifying unit 112 may identify, as the intra-image position in the learning image, the position of the moving object region in the two-dimensional image that corresponds to the earliest time in chronological order from among the multiple two-dimensional images used to generate the learning image.

[0053] As a second modified example, the training image generation unit 111 may generate a two-dimensional image that does not include an afterimage, and then apply blurring to the moving body model in the two-dimensional image to generate a training image that includes an afterimage. Here, blurring is a process that obscures the boundary of the moving body model image. In the blurring process, each pixel on the boundary of the moving body model is processed. Then, the training image generation unit 111 sets the pixel to be processed as the center and calculates the average of the pixel values ​​of the pixel and its surrounding pixels (e.g., nine pixels). Then, the training image generation unit 111 changes the pixel value of the pixel to be processed to this average value. By performing this process on each pixel, a blurred image, i.e., a training image, is obtained. In this way, the training image that includes an afterimage may be generated using 3DCG, and the specific process is not limited to that described in the embodiment.

[0054] In addition, when the training image is generated by blurring processing, the position identification unit 112 identifies the area in the training image that has been blurred, i.e., the area that includes pixels whose pixel values ​​have been changed in the blurring processing, as a moving object area.

[0055] As another example, the position specifying unit 112 may specify, as the intra-image position, the position of a rectangular area including the image of the dynamic body model in the two-dimensional image before the blurring process is performed.

[0056] As another example, the training image generating unit 111 may generate a plurality of two-dimensional images along the movement of the moving object, synthesize these to generate a composite image, and then blur the composite image to generate a training image. Here, the composite image is an image obtained by averaging the pixel values ​​of the plurality of two-dimensional images.

[0057] Alternatively, the training image generating unit 111 may arrange multiple moving body models in one two-dimensional image to generate training data. Alternatively, as another example, the training image generating unit 111 may arrange moving body models at multiple positions.

[0058] As a third modified example, the server device 10 may acquire a moving image showing the movement of a moving object. In this case, the moving object region detection unit 122 detects a moving object region including a moving object in each frame included in the moving image.

[0059] As a fourth variation, server device 10 may acquire a video showing the flight state of the baseball and estimate the flight state of the baseball based on this video. In this case, server device 10 may estimate the flight state based on the position in the image of the moving object region that is the area of ​​the baseball in each frame and the timing of the image capture, that is, based on trajectory information that shows the flight trajectory of the baseball. Here, the flight state includes the initial velocity, number of rotations, axis of rotation, etc. of the baseball.

[0060] An estimation model may be used to estimate the flight state. Here, learning of the estimation model will be described. Server device 10 generates multiple flight states of a baseball that can be thrown by a pitcher, each differing in at least one of the conditions of the initial velocity, the number of rotations, and the axis of rotation. Server device 10 then inputs the flight states and generates trajectory information indicating the positional change of the baseball over time during flight through a physical simulation using a physics simulator. The physics simulator receives the initial values ​​of the velocity, the number of rotations, and the tilt of the axis of rotation of the thrown baseball, and predicts the trajectory of the ball flying under these conditions. Furthermore, server device 10 uses the flight states provided as input and the trajectory information obtained as output for the flight states as a single learning data set, and generates an estimation model that estimates the flight state from the trajectory information through machine learning using the learning data.

[0061] As a fifth modification, the server device 10 may be realized as an information processing system including a plurality of devices. For example, in the information processing system, the learning unit 110 and the detection unit 120 may be configured as different devices.

[0062] According to the program, information processing system, and information processing method of this embodiment configured as described above, the training image generation unit 111 generates two-dimensional training images onto which a moving object model and a background model arranged in a three-dimensional virtual space are projected, the training images including afterimages corresponding to the movement of the moving object model, the position identification unit 112 identifies the position within the training image of a moving object region including at least one image of the moving object model, and the detection model generation unit 113 uses a combination of the training image and the position within the image of the moving object region as training data to generate a detection model that detects a moving object region from a captured image including afterimages corresponding to the movement of the moving object. By using the detection model generated in this manner, it is possible to accurately detect a moving object even when an afterimage is included in the captured image of the moving object.

[0063] Furthermore, according to the program, information processing system, and information processing method of this embodiment, the training image generation unit 111 may generate multiple 2D images in which the dynamic body model is arranged at different positions according to the movement of the dynamic body model, and generate training images based on the multiple 2D images. This allows efficient generation of training data.

[0064] Furthermore, according to the program, information processing system, and information processing method of this embodiment, the position identifying unit 112 may identify the position of an area in the training image that includes multiple moving body models as the position of the moving body area in the image. This makes it possible to automatically identify the position of the moving body area in the image.

[0065] Furthermore, according to the program, information processing system, and information processing method of this embodiment, the training image generation unit 111 may generate a two-dimensional image including a moving body model and a background model arranged in a three-dimensional virtual space, and perform a blurring process on the image of the moving body model included in the two-dimensional image, thereby generating a training image. This allows efficient generation of training images.

[0066] Furthermore, according to the program, information processing system, and information processing method of this embodiment, the position identifying unit 112 may identify the position of a moving object region in an image, including a blurred region, thereby automatically identifying the position of the moving object region in an image.

[0067] Furthermore, according to the program, information processing system, and information processing method of this embodiment, the position identifying unit 112 may identify, as the intra-image position, a position in the training image that corresponds to the position of the moving object region in the two-dimensional image before blurring is performed. This makes it possible to automatically identify the intra-image position of the moving object region.

[0068] Furthermore, according to the program, information processing system, and information processing method of this embodiment, the training image generation unit 111 may generate a plurality of two-dimensional images in which the dynamic body model is arranged at different positions according to the movement of the dynamic body model, generate a composite image based on the plurality of two-dimensional images, and blur the image of the dynamic body model included in the composite image, thereby generating training images. This allows efficient generation of training images.

[0069] Furthermore, according to the program, information processing system, and information processing method of this embodiment, the moving object region may be a rectangular region surrounding at least one image of the moving object model, thereby enabling detection of a moving object region according to the application.

[0070] Furthermore, according to the program, information processing system, and information processing method of this embodiment, the captured image acquisition unit 121 acquires a captured image containing an afterimage corresponding to the movement of a moving object, and the moving object region detection unit 122 detects a moving object region from the captured image acquired by the captured image acquisition unit 121 using a detection model that detects a moving object region containing a moving object from the captured image containing an afterimage corresponding to the movement of the moving object. Here, the detection model is a two-dimensional training image onto which a moving object model and a background model arranged in a three-dimensional virtual space are projected. The training image is generated by generating a training image containing an afterimage corresponding to the movement of the moving object model, identifying the position in the training image of a moving object region containing at least one image of the moving object model, and using a combination of the training image and the position in the image of the moving object region as training data. This makes it possible to accurately detect a moving object even when an afterimage is included in the captured image of the moving object. [Explanation of symbols]

[0071] 1. Information Processing Systems 10 Server device 20 Mobile devices 100 control section 110 study units 111 Learning image generation unit 112 Location identification part 113 Detection model generation unit 120 estimated units 121 Image acquisition unit 122 Moving object area detection unit 130 Communication processing unit 140 Communications Department 150 Storage section 160 Display section 170 Operation section 200 Communications Department 210 Camera 220 Display section

Claims

1. A program that causes a computer to function as a learning image generation unit, a position identification unit, and a detection model generation unit, the training image generation unit generates a two-dimensional training image onto which a moving body model and a background model arranged in a three-dimensional virtual space are projected, the training image including an afterimage according to movement of the moving body model; the position specifying unit specifies an intra-image position of a moving object region including at least one image of the moving object model within the learning image; The detection model generation unit generates a detection model that detects a moving object region from a captured image that includes an afterimage corresponding to the movement of a moving object by using a combination of the training image and the position of the moving object region within the image as training data.

2. 2. The program according to claim 1, wherein the training image generation unit generates a plurality of two-dimensional images in which the dynamic body model is positioned at different positions according to movement of the dynamic body model, and generates the training image based on the plurality of two-dimensional images.

3. The program according to claim 2 , wherein the position specifying unit specifies a position of an area in the training image that includes the plurality of moving body models as the position of the moving body area within the image.

4. 2. The program according to claim 1, wherein the training image generation unit generates a two-dimensional image including a moving body model and a background model arranged in a three-dimensional virtual space, and generates the training image by performing a blurring process on the image of the moving body model included in the two-dimensional image.

5. The program according to claim 4 , wherein the position specifying unit specifies a position in the image of the moving object region that includes the blurred region.

6. The program according to claim 4 , wherein the position specifying unit specifies, as the intra-image position, a position in the learning image that corresponds to a position of the moving object region in the two-dimensional image before the blurring process is performed.

7. The program of claim 1, wherein the training image generation unit generates a plurality of two-dimensional images in which the dynamic body model is positioned at different positions according to the movement of the dynamic body model, generates a composite image based on the plurality of two-dimensional images, and generates the training image by blurring the image of the dynamic body model included in the composite image.

8. The program according to claim 1 , wherein the moving object region is a rectangular region surrounding at least one image of the moving object model.

9. A program that causes a computer to function as a photographed image acquisition unit and a moving object region detection unit, the captured image acquisition unit acquires a captured image including an afterimage according to the movement of a moving object; the moving object region detection unit detects the moving object region from the photographed image acquired by the photographed image acquisition unit by utilizing a detection model that detects a moving object region including a moving object from the photographed image including an afterimage corresponding to the movement of the moving object; The detection model is generating a two-dimensional learning image onto which a moving body model and a background model arranged in a three-dimensional virtual space are projected, the learning image including an afterimage according to the movement of the moving body model; Identifying an intra-image position within the training image of a moving object region including at least one image of the moving object model; The program is generated by using a combination of the training image and the position of the moving object region within the image as training data.

10. a training image generation unit that generates a two-dimensional training image onto which a moving body model and a background model arranged in a three-dimensional virtual space are projected, the training image including an afterimage according to the movement of the moving body model; a position specifying unit that specifies a position within the training image of a moving object region that includes at least one image of the moving object model; a detection model generation unit that generates a detection model for detecting a moving object region from a captured image including an afterimage corresponding to the movement of the moving object by using a combination of the training image and the position of the moving object region in the image as training data; An information processing system comprising:

11. a captured image acquisition unit that acquires a captured image including an afterimage according to the movement of a moving object; a moving object region detection unit that detects a moving object region including a moving object from a photographed image that includes an afterimage corresponding to the movement of the moving object, using a detection model for detecting the moving object region from the photographed image acquired by the photographed image acquisition unit; Equipped with The detection model is generating a two-dimensional learning image onto which a moving body model and a background model arranged in a three-dimensional virtual space are projected, the learning image including an afterimage according to the movement of the moving body model; Identifying an intra-image position within the training image of a moving object region including at least one image of the moving object model; The information processing system generates the training data by using a combination of the training image and the position of the moving object region within the image as training data.

12. An information processing method performed by a computer having a control unit, a step of generating, by the control unit, a two-dimensional learning image onto which a moving body model and a background model arranged in a three-dimensional virtual space are projected, the learning image including an afterimage according to the movement of the moving body model; a step of the control unit specifying an intra-image position within the training image of a moving object region including at least one image of the moving object model; a step in which the control unit generates a detection model for detecting a moving object region from a captured image including an afterimage corresponding to movement of the moving object by using a combination of the learning image and the position in the image of the moving object region as learning data; An information processing method, including:

13. An information processing method performed by a computer having a control unit, a step of the control unit acquiring a photographed image including an afterimage according to the movement of a moving object; a step in which the control unit detects a moving object region from the acquired photographed image using a detection model that detects a moving object region including a moving object from the photographed image including an afterimage corresponding to the movement of the moving object; Including, The detection model is generating a two-dimensional learning image onto which a moving body model and a background model arranged in a three-dimensional virtual space are projected, the learning image including an afterimage according to the movement of the moving body model; Identifying an intra-image position within the training image of a moving object region including at least one image of the moving object model; The information processing method generates the training data by using a combination of the training image and the position of the moving object region within the image as training data.

Images