Information processing method, program, and information processing device

Abstract

From an image containing multiple people, the composition is arranged to show the main figures according to the type of event depicted in the image. This invention provides an information processing method that can generate images primarily based on [a specific element]. [Solution] An information processing method according to one embodiment of the present disclosure is used to process an overall image showing multiple people. The system acquires the event type of the overall image and identifies the event type of the multiple events according to the identified event type. Identify the subject from the person, determine the area containing the subject as the trimming area, and the trimming Generate individual images by extracting the smear region.

Description

Technical Field

[0001] This technology relates to an information processing method, a program, and an information processing apparatus.

Background Art

[0002] Conventionally, technologies for editing images have been proposed. For example, the trimming device described in Patent Document 1 trims an image including a person's face to a panorama size.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the technology described in Patent Document 1 does not consider generating an image in which a main person corresponding to the type of event depicted in the image is the subject of the composition from an image in which a plurality of people are depicted.

[0005] In view of such circumstances, the present disclosure has been made, and an object thereof is to provide an information processing method or the like capable of generating an image in which a main person corresponding to the type of event depicted in an image is the subject of the composition from an image in which a plurality of people are depicted.

Means for Solving the Problems

[0006] An information processing method according to an embodiment of the present disclosure acquires an overall image in which a plurality of people are depicted, identifies the event type of the overall image, and selects from the plurality of people according to the identified event type. ​​​​​Identify the subject, determine the area containing the subject as the trimming area, and the trimming area Generate individual images by extracting them. [Effects of the Invention]

[0007] In an information processing method according to one embodiment of the present disclosure, from an image showing multiple people, the image It is possible to generate images with the main figures in the composition based on the type of event depicted. be. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing an example of the configuration of an information processing device. [Figure 2] This is an explanatory diagram showing an example of a complete image database. [Figure 3] This is an explanatory diagram showing an example of an individual image database. [Figure 4] This is an explanatory diagram showing an example of an individual image condition database. [Figure 5] This is an explanatory diagram illustrating an example of an event type identification model. [Figure 6] This is an explanatory diagram showing an example of a detection model. [Figure 7] This is an explanatory diagram showing an example of a target identification section. [Figure 8] This is an explanatory diagram showing an example of a trimming section. [Figure 9] This is an explanatory diagram showing an example of a super-resolution processing model. [Figure 10] This is an explanatory diagram illustrating an example of a segmentation model. [Figure 11] This is an explanatory diagram showing an example of a blurring processing unit and a synthesis unit. [Figure 12] This is an explanatory diagram showing an example of an individual image display screen. [Figure 13] This flowchart shows an example of individual image output processing. [Figure 14] This is an explanatory diagram showing an example of a detection model according to Embodiment 2. [Figure 15] It is an explanatory diagram showing an example of the target person specifying unit according to Embodiment 2. [Figure 16] It is an explanatory diagram showing an example of the trimming unit according to Embodiment 2. [Figure 17] It is an explanatory diagram showing an example of the detection model according to Embodiment 3. [Figure 18] It is an explanatory diagram showing an example of the target person specifying unit according to Embodiment 3. [Figure 19] It is an explanatory diagram showing an example of the trimming unit according to Embodiment 3.

Mode for Carrying Out the Invention

[0009] (Embodiment 1) The information processing apparatus 1 according to Embodiment 1 specifies the event type of the overall image in which a plurality of persons are captured, specifies a target person from the plurality of persons according to the specified event type, determines a region including the target person as a trimming region, and generates an individual image obtained by cutting out the trimming region. Hereinafter, Embodiment 1 will be described with reference to the drawings. and specifies a target person from a plurality of persons according to the specified event type, determines a region including the target person as a trimming region, and generates an individual image obtained by cutting out the trimming region. Hereinafter, Embodiment 1 will be described with reference to the drawings. Embodiment 1 will be described with reference to the drawings.

[0010] FIG. 1 is a block diagram showing a configuration example of the information processing apparatus 1. The information processing apparatus 1 is, for example, a personal computer, and includes a processing unit 11, a storage unit 12, a communication unit 13, an input unit 14, and a display unit 15. Note that the information processing apparatus 1 may be a smartphone, a tablet terminal, or a wearable device or the like. The processing unit 11 is composed of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), or a quantum processor or the like, and reads and executes a program P (program product) and a database stored in advance in the storage unit 12 to perform various controls. Unit), an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), or a quantum processor or the like, and reads and executes a program P (program product) and a database stored in advance in the storage unit 12 to perform various controls. and reads and executes a program P (program product) and a database stored in advance in the storage unit 12 to perform various controls. It performs processing, calculations, etc. Furthermore, a database server, etc., is provided outside the information processing device 1. The database may be read from a database server or the like. Furthermore, the information processing device 1, The functionality may be implemented by multiple server devices or computers. Furthermore, the information processing device 1 may correspond to a node on the blockchain.

[0011] The storage unit 12 of the information processing device 1 is, for example, a volatile memory and a non-volatile memory. Memory section 12 contains program P, overall image DB121, individual image DB122, and individual image conditions. DB123, Event Type Identification Model M1, Detection Model M2, Super-Resolution Processing Model M3, It also stores the segmentation model M4. Note that program P is a computer. The data may be provided to the information processing device 1 using a storage medium 12a in which the data is stored in a readable format. The storage medium 12a is, for example, portable memory. An example of portable memory is a CD-RO M, USB (Universal Serial Bus) memory, SD card, microSD card or Examples include compact flash memory (registered trademark). The storage medium 12a is a portable memory. In some cases, the processing element of the processing unit 11 uses a reading device (not shown) to read data from the storage medium 12a. The program P may be read. The read program P is written to the memory unit 12. Furthermore, program P communicates with an external device via the communication unit 13, thereby enabling the information processing device 1 May be provided to: Overall image DB121, Individual image DB122, Event type specific model Model M1, detection model M2, super-resolution processing model M3, and segmentation model M4 Details will be provided later. Event type identification model M1, detection model M2, super-resolution processing The logic model M3, or segmentation model M4, is different from the information processing device 1. The image is stored on the server or computer, and the information processing device 1 processes the image on the server or computer. The data may be sent to a server or computer where various processes are performed.

[0012] The communication unit 13 is a communication module or for communicating with the imaging device 2 by wire or wireless. It is a communication interface, such as wide-area wireless communication like LTE (registered trademark), 4G, or 5G. This is a communication module. The processing unit 11 communicates via the communication unit 13, for example, with the internet. Communication with the imaging device 2 is performed via an external network N.

[0013] The input unit 14 receives input from the user, for example, an instruction to display an individual image. The input unit 14 may also accept input for instructions to generate individual images. For example, the input unit 14 It consists of a keyboard or mouse, etc.

[0014] The display unit 15 displays the overall image acquired from the imaging device 2, as well as the generated individual images. The display unit 15 may be, for example, a liquid crystal panel, an organic EL (Electro-Luminescence) display, etc. It is composed of displays.

[0015] The imaging device 2 is composed of, for example, a camera. The imaging device 2 is used for, for example, school events. , taking group photos including multiple people at events such as track and field competitions or concerts, The captured overall photograph is transmitted to the information processing device 1. The camera 2 then transmits the captured overall photograph USB memory, SD card, microSD card, or CompactFlash memory (registered The information processing device 1 may store the data on a portable storage medium such as a registered trademark. By reading from the storage medium, it is possible to obtain a complete photograph. Furthermore, information processing... Device 1 may include a camera unit, and the camera unit may take an overall photograph. Information Processing Device 1 If the device is a smartphone, tablet, or wearable device, the information The camera function of the processing device 1 may take a picture of the whole thing. The types of events are not limited to school events, track and field events, or concerts. This could include a sports competition, a group trip, or a performance such as a dance or martial arts demonstration.

[0016] Figure 2 is an explanatory diagram showing an example of the overall image DB121. The overall image DB121 contains all Information about the body image is stored here. The management items (fields) of the overall image DB121 are all Body image ID field, shooting date and time field, overall image field, and event type field It includes a field. The shooting date and time field stores the date and time the entire image was taken. The overall image ID field contains a number (ID) assigned to distinguish the overall image. It is stored. The overall image field contains the overall image acquired by the processing unit 11 from the imaging device 2. The image is stored, for example, in a file format. The event type field contains the event described later. The event captured in the overall image was identified by the type identification model M1 (see Figure 5). The event type is stored. Note that the event type field contains the information entered by the user. The event type may also be stored. In addition, the overall image DB121 contains the overall image taken. A shooting location field may be provided to store the location.

[0017] Figure 3 is an explanatory diagram showing an example of the individual image DB122. The individual image DB122 contains all Information about individual images generated from body images is stored. Individual image DB122 The fields are: Individual Image ID field, Individual Image field, Subject ID field Includes field and overall image ID field. Individual image ID field contains individual image A number (ID) assigned to distinguish them is stored. The individual image field contains: Individual images (edited images) after various processing steps are performed are stored, for example, in a file format. The ID field contains the number assigned to the person (subject) who is primarily depicted in the individual image. The ID is stored. That is, the processing unit 11 detects the detection model M2 (see Figure 6), which will be described later. Based on the subject's features detected by ), individual features of the same subject are extracted from multiple overall images. If images are generated, the same subject ID is associated with each of the multiple individual images. Overall Image I The D field stores the overall image ID of the overall image from which the individual images were generated. ru.

[0018] Figure 4 is an explanatory diagram showing an example of the individual image condition DB123. Individual Image Condition DB123 This includes the target audience, non-target audience members, and trimming area, depending on the event type. The number of items is stored. The management items (fields) of the individual image condition DB123 are, for example, Vent type field, target person field, non-target person field, and trimming area field Includes fields. The event type field stores the event type. Target field The field stores the criteria for eligible individuals, according to the event type. (Non-eligible individuals field) The field stores the conditions for individuals who are not eligible, depending on the event type. The trimming area field contains trimming area conditions according to the event type. It will be delivered. In addition, depending on the type of event, the target people, non-target people, and trimming will be determined. The conditions for the G region may be those incorporated into program P.

[0019] For example, in a place where the event type is "Activities at a school, nursery school, or kindergarten (school, etc.)" In summary, the target group is "children," the non-target group is "adults," and the trimming area is "primarily children, but the entire body." It is a region that has been cut out to accommodate the following. If the event type is track and field, the target group "A person wearing a bib number" is a person who is not eligible, "A person not wearing a bib number" is a person who is not eligible, and Trimi The cutting area is designed so that the area on the side of the subject's direction of movement is wider than the area on the side not moving. It is a "reclaimed territory." If the event type is a concert, the target audience is "those who play musical instruments." "People who are present," non-target individuals are "people who are not playing a musical instrument," and the trimming area is "target individuals and This is "an area cut out to accommodate the instrument that the subject will be playing."

[0020] Figure 5 is an explanatory diagram showing an example of the event type identification model M1. Dell M1 supports, for example, CNN (Convolutional Neural Network) and R-CNN (Regions-C). NN), Fast R-CNN, Faster R-CNN, SSD (Single Shot Multib This includes neural networks such as Look Detector or YOLO (You Only Look Once). It is composed of object detection machine learning models. The event type identification model M1 is image Using training data that associates images with image event types, when a whole image is input... It is trained to output the event type.

[0021] The event type identification model M1 is a model that includes neural networks such as CNNs. When configured, the event type identification model M1 accepts input of pixel values ​​for the entire image. It has a number of neurons and passes the input pixel values ​​to the hidden layer. The hidden layer is the image of the whole image. It has multiple neurons that extract features and passes the extracted image features to the output layer. The force layer outputs the event type of the overall image based on the image features. In this embodiment, The overall image shows multiple children, and the event type identification model M1 identifies the overall image Output "Activities at schools, daycare centers, or kindergartens (schools, etc.)" as the event type. Oh, the event type identification model M1 further subdivides events such as sports days, field trips, or cultural festivals. The events that have been recorded may be output as event types. Also, the processing unit 11 of the information processing device 1, By inputting the entire image into the VLM (Vision Language Model) autoencoder... The event type may be identified by the resulting image embedding vector.

[0022] Figure 6 is an explanatory diagram showing an example of detection model M2. The processing unit 11 detects the entire image using the detection model By inputting into the Dell M2, it detects people in the overall image. Also, processing unit 1 1 identifies target and non-target individuals using the target identification unit 111. Detection model M2 is For example, CNN, R-CNN, Fast R-CNN, Faster R-CNN, SSD, Alternatively, it can be composed of an object detection machine learning model that includes a neural network such as YOLO. The detection model M2 uses an image and regions in the image that contain people, as well as the attributes of the people. Using the associated training data, when a whole image is input, the region containing a person and the person's attributes are identified. It outputs the type. Note that the detection model M2 is a model that is integrated with the event type identification model M1. It may be configured as follows. Detection model M2 includes children, adults, competitors, non-competitors, performers, and This outputs attributes based on the category of a person or their actions, such as non-performers.

[0023] In the case where the detection model M2 is composed of a model that includes a neural network such as a CNN, In addition, the detection model M2 has multiple neurons that accept input of pixel values ​​of the whole image, The extracted pixel values ​​are passed to the intermediate layer. The intermediate layer extracts image features from the overall image using multiple layers. It has a neuron and passes the extracted image features to the output layer. The output layer is based on the image features. The system outputs the area in the overall image where a person is visible. The output of the detection model M2 shown in Figure 6 In images showing force, the area including the person is indicated by a solid bounding box. It can be done.

[0024] Figure 7 is an explanatory diagram showing an example of the target identification unit 111. The target identification unit 111 processes As a functional unit of section 11, it performs the processing described later. Processing unit 11 then receives the target identification unit 111. The input is an overall image with the area including the person output. The subject identification unit 111 determines the event type Depending on the event type output by a specific model M1, refer to the individual image condition DB123, and Identify non-target individuals based on the attributes of objects (identification based on attributes). As shown in Figure 7, event If the activity type is school-related, the subject identification unit 111 identifies adults among the detected individuals. Identify non-target individuals. In the overall image after identification based on attributes, the range including non-target individuals is identified. This is indicated by a dashed bounding box. From here on, the persons identified as non-targets The bounding box is indicated by a dashed line, and the bounding box of the person identified as the subject is shown. The box is indicated by a solid line.

[0025] The target identification unit 111 identifies individuals who are overlapping with other people or objects in the foreground as non-target individuals. Identify. The target identification unit 111 will not identify a person whose face is obscured by another person or object. It may be used to identify the target person. Also, the target identification unit 111 is such that the bounding box is the overall picture The figures touching the edges of the image, the figures that are out of focus, and the bounding box Individuals whose size is less than a predetermined size (for example, 2% of the overall image size) will be excluded. Identify (exclude from target group) (identification based on how the image appears). Target group identification unit 111 (processing unit 11 ) refers to the size, clarity, or bounding of the part of the child's whole body that is visible in the overall image. Non-target individuals are identified by evaluating the size of the box. Among the individuals, those who were not deemed ineligible through the above-mentioned process are identified as eligible. .

[0026] Figure 8 is an explanatory diagram showing an example of the trimming unit 112. The trimming unit 112 processes The functional part of section 11 executes the processing described later. The trimming section 112 is the event type Depending on the event type output by a specific model M1, refer to the individual image condition DB123, and individually Determine the cropping area for the image. As shown in Figure 8, the event type is school activities, etc. In that case, the trimming area is the region where the entire body of the child is included. The decision is made. The conditions for the trimming area are the bounding box in each individual image. The ratio of the surrounding area (excess) to the subject, or the presence of persons other than the subject in individual images. It may also be determined by the proportion of the area or the number of people, etc. People appearing in the overall image If multiple individuals are identified as subjects, the cropping area will be extracted, and the subjects will be identified. Individual images related to each are generated. The trimming unit 112 processes each of the generated (output) individual images. Each image will be assigned an individual image ID.

[0027] Figure 9 is an explanatory diagram showing an example of a super-resolution processing model M3, etc. The processing unit 11 is an individual image The image is input to the resizing unit 113, and the image size of the individual images is reduced (resizing process). Size processing can speed up the processing described later.

[0028] The processing unit 11 inputs the resized individual images into the super-resolution processing model M3. The super-resolution processing model M3 is, for example, a Generative Adversarial Network (GAN). It consists of machine learning models that perform image generation. Note that the super-resolution processing model is SRCNN (Super-Resolution CNN), Pix2pix, or CUT (Contrastive Learning) Machine learning models such as ng for Unpaired Image-to-Image Translation, or VLM. It may be composed of generative models such as the above. The super-resolution processing model M3 is composed of GANs If configured, the super-resolution processing model M3 includes a generator and a judge. The generator of the M3 model generates high-resolution images based on the features of individual images. The machine compares the image generated by the generator with the individual input images, and the generated image The generator determines the authenticity of each individual image. The generator obtains the determination result from the judge and the obtained judgment The image is regenerated based on the result. The generator will regenerate the image if the judgment result output by the classifier is "true". Images are generated until the result is "true", and the super-resolution processing model M3 then super-processes the image that is deemed "true" into a super-resolution image. Output as an image.

[0029] Figure 10 is an explanatory diagram showing an example of the segmentation model M4. The processing unit 11 is The super-resolution image output by the super-resolution processing model M3 is input to the segmentation model M4. The segmentation model M4 is, for example, an FCN (Fully Convolutional Network). Semantic segmentation such as seg-net or FPN (Feature Pyramid Networks) It consists of machine learning models with segmentation capabilities. M4 is an image and a label indicating whether or not each pixel in the image represents a subject. Using training data that associates each pin, when an individual image (super-resolution image) is input, each pin Whether or not a pixel represents a subject, that is, the pixel that represents a subject in an individual image. Outputs the area of ​​the character (person area).

[0030] The segmentation model M4 is a model that includes neural networks such as FCN. When configured by a segmentation model M4, it receives input of pixel values ​​from the overall image. It has multiple neurons that receive input and pass the pixel values ​​to the hidden layer. It has multiple neurons that extract image features from an image, and the extracted image features are received by the output layer. The output layer outputs the semantic range of the subject based on the image features. In the example shown in Figure 10, the segmentation model M4 individually divides the subject's personal domains. Extract and output from the image.

[0031] Figure 11 is an explanatory diagram showing an example of a blurring processing unit 114 and a blending unit 115. The processing unit 114 and the synthesis unit 115, as functional units of the processing unit 11, execute the processes described later. The processing unit 11 receives individual images that have not undergone super-resolution processing as input to the blur processing unit 114. The blurring processing unit 114 made the individual images out of focus and blurred (blurring processing was performed) Outputs a blurred image.

[0032] The synthesis unit 115 of the processing unit 11 processes the blurred image and segmented image output by the blurring processing unit 114. The human region output by the M4 phonography model was acquired and superimposed onto the blurred image. Outputs a collection of images. The person region is the position of each pixel in the individual image that is included in the person region. By matching the information with the position of each pixel in the blurred image, the information superimposed on the blurred image is This allows the processing unit 11 to blur the background area other than the person area in the individual image. Outputs the edited image after processing.

[0033] Figure 12 is an explanatory diagram showing an example of an individual image display screen. The processing unit 11 is connected to the display unit 15. The individual images (edited images) that have undergone the various processes described above are displayed. The processing unit 11 also performs individual Along with the individual images, the overall image from which the individual images were generated is displayed on the display unit 15. The processing unit 11 performs each processing step for the overall image or individual images shown in Figures 5 to 11. The image may be displayed on the display unit 15. In addition, the processing unit 11 may display the individual image on the individual image display screen. It receives input for instructions to output individual images to other terminals or to print devices. You may kick it.

[0034] Figure 13 is a flowchart showing an example of individual image output processing. The processing unit 11 reads the overall image from the overall image DB 121 (S1). The processing unit 11 reads the overall The image is input to the event type identification model M1 (S2), and the event type is output (S3). The processing unit 11 inputs the overall image to the detection model M2 (S4) and detects people in the overall image. Output the included area and the attributes of the person (S5).

[0035] The processing unit 11 of the information processing device 1 processes the event type output by the event type identification model M1. Depending on the individual image conditions DB123, the conditions for subjects, non-subjects, and trimming areas are selected. The data is read (S6). The processing unit 11 reads the data based on the attributes of the person output by the detection model M2. Individuals who meet the criteria for being a target person are excluded from the target group (designated as non-target persons) (S7). Processing Unit 11 is a person in the overall image who is in the foreground and overlapping with another person or object. The person is excluded from the subjects (S8). The processing unit 11 checks if the bounding box is the overall image. People touching the edge, people experiencing out-of-focus shots, and large bounding boxes Individuals whose size is less than a predetermined size (for example, 2% of the overall image size) will be excluded from the subjects. Remove (S9). The processing unit 11 determines the conditions for the trimming area based on the event type of the overall image. The trimming area is determined accordingly (S10). The processing unit 11 cuts out the trimming area. Individual images are generated (S11).

[0036] The processing unit 11 performs a resizing process on the individual images (S12). The processing unit 11, Individual images are input to the super-resolution processing model M3 (S13), and a super-resolution image is output (S14). The processing unit 11 inputs the super-resolution to the segmentation model M4, and in the super-resolution image... Extract and output the person region of the subject (S15).

[0037] The processing unit 11 performs blurring on individual images (S16). The processing unit 11 performs blurring on individual images. By superimposing the human region extracted from the super-resolution image onto the individual images that have undergone processing... Output the edited image (S17).

[0038] The processing unit 11 stores the output edited image in the individual image DB 122 (S18). Then, the processing unit 11 displays the edited image on the display unit 15 (S19) and terminates the process.

[0039] According to the configuration and processing of this embodiment, from an image containing multiple people, the event in the image Identify key figures based on the type of project and generate images with each figure as the main subject of the composition. It is possible to further highlight the subject through super-resolution processing and blurring. It is possible to generate individual images.

[0040] (Embodiment 2) In Embodiment 2, the target audience is when the event type of the overall image is track and field. This section explains the identification of the individual image and the generation of individual images. Individual image conditions DB123 (see Figure 4) are shown. As shown above, if the event type in the overall image is track and field, the target person is "wearing a bib number." "Persons who are being filmed," non-target individuals are "people without bibs," and the trimming area is "the progress of the subject." This is a region that has been cut out such that the region on the direction of travel is larger than the region on the non-traveling direction side. .

[0041] Figure 14 is an explanatory diagram showing an example of the detection model M2 according to Embodiment 2. The detection model M2 uses training data that associates images with the position of bibs in those images. Using this method, when a full image is input, the system outputs (detects) the region containing the person and the position of the bib. It is trained to do so. Furthermore, the detection model M2 detects uniforms, hats, and other items related to the competition. Gloves, shoes, or other athletic equipment may be detected.

[0042] Figure 15 is an explanatory diagram showing an example of the target identification unit 111 according to Embodiment 2. Embodiment The subject identification unit 111 related to 2 identifies, among the persons detected by the detection model M2, Individuals without the "n" symbol are identified as non-target individuals. Furthermore, the target identification unit 111 is related to the competition. Persons not wearing a uniform, hat, glove, or shoes, or persons involved in the competition Individuals who do not possess the tools may be identified as non-target individuals. Furthermore, the target identification unit 111, Similar to Embodiment 1, another person or object is in the foreground, overlapping person, bounding The box is touching the edge of the overall image, the person is out of focus, and the boundary The size of the box is less than a predetermined size (for example, 2% of the size of the overall image). Identify individuals as non-targets.

[0043] Figure 16 is an explanatory diagram showing an example of a trimming section 112 according to Embodiment 2. Embodiment The trimming unit 112 related to 2 refers to the individual image condition DB and the area on the side of the subject's direction of movement. However, the area that is cut out to be wider than the area on the non-moving side is determined to be the trimming area. The trimming area should be wider on the side of the subject's direction of movement than on the side of the subject's direction of movement. By cropping the area in a specific way, the sense of dynamism of the subject in each individual image is highlighted. It is done. In the example shown in Figure 16, the processing unit 11 is based, for example, on the orientation of the subject's face. The direction of movement of the subject is identified as to the left. The trimming unit 112 is bounding box The region Rf on the left side (the direction of the subject's movement) relative to the 'ks' region is on the right side (the direction of the subject's non-movement) The trimming area is determined to be wider than the Rb region.

[0044] According to the processing according to this embodiment, the athletes are identified as the target from an overall image of a track and field event. It is possible to generate individual images.

[0045] (Embodiment 3) In Embodiment 3, when the event type of the overall image is a concert, the target audience This section explains the generation of specific and individual images. Individual image conditions are shown in DB123 (see Figure 4). For example, if the event type for the overall image is a concert, the subject is "a person playing a musical instrument." ", Non-target individuals are "people who do not play musical instruments", and the trimming area is "Target individuals and target individuals It is "an area cut out to accommodate the instrument being played."

[0046] Figure 17 is an explanatory diagram showing an example of the detection model M2 according to Embodiment 3. The detection model M2 uses training data that associates images with the positions of musical instruments in those images. When a full image is input, the system will output (detect) the area containing people and the position of musical instruments. It is trained on. Furthermore, detection model M2 detects the performer's clothing or tie, etc. That's fine.

[0047] Figure 18 is an explanatory diagram showing an example of the target identification unit 111 according to Embodiment 3. Embodiment The person identification unit 111 related to 3 identifies the person who played the instrument among the person detected by the detection model M2. Individuals who are not performing are identified as non-target individuals. The target identification unit 111 also identifies the performers' costumes. Alternatively, individuals not wearing ties or similar attire may be identified as non-target individuals. The fixed part 111, similar to Embodiment 1, is a person who is overlapping another person or object in front of them. A person whose bounding box touches the edge of the overall image, or a person who is out of focus. , and the size of the bounding box is a predetermined size (for example, 2 times the size of the overall image) Individuals with a percentage below %) are identified as ineligible.

[0048] Figure 19 is an explanatory diagram showing an example of a trimming section 112 according to Embodiment 3. Embodiment The trimming unit 112 related to 3 refers to the individual image condition DB and the subject and the performance performed by the subject. The area that is cut out to accommodate the instrument is determined as the trimming area. By creating an area that is cut out to accommodate the subject and the instrument they play, It is possible to generate individual images that show the person performing the instrument.

[0049] According to the processing according to this embodiment, the performers are identified as subjects from the overall image of the concert. It is possible to generate individual images.

[0050] The embodiments disclosed herein are illustrative in all respects and not restrictive. It should be possible to combine the technical features described in each embodiment with each other. The scope of this invention is limited to all modifications within the claims and the scope equivalent to the claims. It is intended that this be included. Also, the independent claims and dependent claims described in the claims. The terms, regardless of the format of the citation, should be combined with each other in all possible combinations. This is possible. Furthermore, the claims may include claims that refer to two or more other claims. The format used is the multi-claim format, but it is not limited to this format. Include multiple claims (multi-multi-claims) that cite at least one claim. You may also use a specific format when writing it. [Explanation of symbols]

[0051] 1: Information Processing Device 11: Processing Section 111: Target Identification Department 112: Trimming section 113: Resizing section 114: Blur processing 115: Synthesis part 12: Storage section 12a:Storage medium 13: Communications Department 14: Input section 15:Display section 121: Overall Image Database 122: Individual Image Database 123: Individual Image Conditions Database M1: Event type identification model M2: Detection Model M3: Super-resolution processing model M4: Segmentation Model 2: Imaging device N: Network P: Program

Claims

1. Obtain a full image containing multiple people, Identify the event type of the aforementioned overall image Depending on the identified event type, the target person is selected from the aforementioned multiple individuals. The area including the aforementioned subject is determined to be the trimming area. Generate individual images by cutting out the aforementioned trimmed area. Information processing methods.

2. If the aforementioned event type is an activity at a school, nursery school, or kindergarten, adults will be considered as the target group. Identify non-target individuals who are not the target person. The information processing method according to claim 1.

3. If the event type is track and field, then the person without a bib number is considered the subject. Identify ineligible individuals. The information processing method according to claim 1.

4. Identify individuals who are overlapped by other people or objects as non-target individuals. The information processing method according to any one of claims 1 to 3.

5. In the aforementioned overall image, individuals whose size is less than a predetermined size are considered as subjects. Identify ineligible individuals. The information processing method according to any one of claims 1 to 3.

6. The trimming area is determined according to the type of event. The information processing method according to any one of claims 1 to 3.

7. Super-resolution processing is performed on the aforementioned individual images using a machine learning model. The information processing method according to claim 1.

8. After performing a resizing process on the individual images, super-resolution processing is performed. The information processing method according to claim 7.

9. The background area of ​​the individual image, excluding the area containing the subject, is blurred. Output the processed and edited image. The information processing method according to claim 7 or 8.

10. The person region is extracted from the individual images that have undergone super-resolution processing. A blurring process is performed on the aforementioned individual images. The extracted area of ​​the person is superimposed onto the blurred individual image. By doing so, the edited image is output. The information processing method according to claim 9.

11. Obtain a full image containing multiple people, Identify the event type of the aforementioned overall image Depending on the identified event type, the target person is selected from the aforementioned multiple individuals. The area including the aforementioned subject is determined to be the trimming area. Generate individual images by cutting out the aforementioned trimmed area. A program that instructs a computer to perform a process.

12. Obtain a full image containing multiple people, Identify the event type of the aforementioned overall image Depending on the identified event type, the target person is selected from the aforementioned multiple individuals. The area including the aforementioned subject is determined to be the trimming area. Generate individual images by cutting out the aforementioned trimmed area. Processing unit that executes the process An information processing device equipped with the following features.

Images