Automatic trimming program, automatic trimming device, and automatic trimming method

Abstract

To allow for automatic trimming in consideration of the intention of a person who has shot an image.SOLUTION: An automated trimming program includes: acquiring image data for trimming; detecting predetermined objects included in the image data by image recognition processing; specifying an essential object which must be fit within a template frame, out of the detected objects; setting a template frame on the basis of a predetermined rule for the image data; executing zooming processing of the image data so that the essential object may be fit within the template frame; executing moving processing of the image data with respect to the template frame on the basis of a predetermined reference including references of top, bottom, left and right margins with respect to the essential object in the template frame; and outputting an inside area of the template frame from the image data, as a trimming image, in a fixed positional relationship.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for automatically trimming image data acquired by photography so that a subject fits within a frame of a predetermined size. [Background technology]

[0002] Conventionally, there exists a business model of a photo studio where skilled photographers take photos and provide them to customers. In recent years, it has become common to take photos digitally and then appropriately correct the photos. One of the correction processes is a trimming process that cuts the image data of the photo into a predetermined template frame to adopt a composition that makes the subject look the best.

[0003] For example, Patent Document 1 discloses an automatic trimming technology in which a still image of a subject is captured at a wide angle, a mask pattern of the subject is generated from the captured image, skin-colored portions are extracted from the captured image, and the mask pattern in which skin-colored portions exist is detected as a human pattern, and a portion of the captured image corresponding to the mask pattern detected as the human pattern is cut out, and the cut-out partial image is enlarged to a predetermined image size. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-061114 A Summary of the Invention [Problem to be solved by the invention]

[0005] The above-mentioned Patent Document 1 is a technology that automatically captures a wide-angle image in a fixed shooting range and then trims only the area in which the subject is captured. Although it is possible to detect the area of ​​the subject and perform trimming, there is a problem that the processing is uniform. In addition, when an object other than a person is captured, it is not possible to determine how to handle the object when trimming. In other words, when the technology of Patent Document 1 is applied to an image that is not automatically captured but is intentionally captured by the photographer, there is a problem that it is not possible to perform trimming that takes into account the photographer's intention.

[0006] The present invention has been made in consideration of the above problems, and has an object to provide an automatic trimming program, an automatic trimming device, and an automatic trimming method that enable automatic trimming that takes into account the photographer's intentions. [Means for solving the problem]

[0007] The automatic trimming program of the present invention is an automatic trimming program for causing a computer to realize a process of automatically trimming image data obtained by photographing so that a subject fits within a template frame of a predetermined size, and is characterized in that the automatic trimming program causes the computer to realize an image data acquisition function for acquiring target image data, a detection function for detecting a predetermined object appearing in the image data by image recognition processing, an identification function for identifying a required object that must be fitted within the template frame from the detected objects, a frame setting function for setting the template frame on the image data based on predetermined rules, a scaling function for performing a scaling process on the image data so that the required object fits within the template frame, a movement function for performing a movement process of the image data relative to the template frame based on predetermined criteria including criteria for the amount of margins above, below, left and right for the required object within the template frame, and a trimming function for outputting the inner area of ​​the template frame from the image data in a determined positional relationship as a trimmed image.

[0008] In addition, the automatic trimming program of the present invention is characterized in that the identification function further includes a function for identifying an acceptable object that is allowed to fit within the template frame, and the zoom function and the movement function perform processing to allow the acceptable object to fit within the frame within a range that does not satisfy specified prohibition conditions.

[0009] In addition, the automatic trimming program of the present invention is characterized in that the zoom function sets a judgment frame that is a predetermined size larger than the size of the template frame at the initial position first set in the image data, and when the permissible object is contained inside the judgment frame, a zoom process is performed so as to contain the permissible object within the template frame.

[0010] In addition, the automatic trimming program of the present invention is characterized in that the identification function further includes a function for identifying prohibited objects that should not be contained within the template frame, and the zoom function performs a zoom process so that the prohibited objects do not remain within the template frame to the extent that does not satisfy specified prohibition conditions.

[0011] The automatic trimming device of the present invention is an automatic trimming device for automatically performing a process of trimming image data obtained by photographing so that a subject fits within a template frame of a predetermined size, and is characterized in that it comprises an acquisition unit that acquires the target image data, a detection unit that detects a predetermined object appearing in the image data by image recognition processing, an identification unit that identifies a required object that must be fitted within the template frame from the detected objects, a frame setting unit that sets the template frame on the image data based on predetermined rules, a scaling unit that performs a scaling process on the image data so that the required object fits within the template frame, a movement unit that performs a movement process of the image data relative to the template frame based on predetermined criteria including criteria for the amount of margins above, below, left and right for the required object within the template frame, and a trimming unit that outputs the inner area of ​​the template frame from the image data in the determined positional relationship as a trimmed image.

[0012] The automatic trimming method of the present invention is an automatic trimming method that uses a computer to automatically trim image data obtained by photography so that a subject fits within a template frame of a predetermined size, and is characterized in that it includes an acquisition step for acquiring target image data; a detection step for detecting a predetermined object appearing in the image data by image recognition processing; an identification step for identifying a required object that must be fitted within the template frame from the detected objects; a frame setting step for setting the template frame on the image data based on predetermined rules; a scaling step for scaling the image data so that the required object fits within the template frame; a movement step for moving the image data relative to the template frame based on predetermined criteria including criteria for the amount of margins above, below, left and right for the required object within the template frame; and a trimming step for outputting the inner area of ​​the template frame from the image data in the determined positional relationship as a trimmed image. Effect of the Invention

[0013] According to the present invention, it is possible to automatically perform trimming that includes all required objects and sets appropriate margins. Also, it is possible to recognize an allowable object included in a template frame as a subject included based on the photographer's intention, and to allow the object to fit within the frame while respecting the photographer's intention. Also, by using a judgment frame, it is possible to fit an allowable object that is presumed to be intended by the photographer to be included in the image data within the template frame, and trimming that respects the photographer's intention is possible. Also, by performing a scaling process so that prohibited objects do not remain within the template frame, it is possible to perform automatic trimming so that objects set as prohibited objects do not remain within the template frame. [Brief description of the drawings]

[0014] [Figure 1] 1 is a block diagram showing a configuration of an automatic trimming device 10 according to the present invention. [Diagram 2] FIG. 2 is a flow chart showing the flow of automatic trimming processing in the automatic trimming device 10 according to the present invention. [Diagram 3] FIG. 2 is a flow chart showing the flow of enlargement / reduction processing in the automatic trimming device 10 according to the present invention. [Figure 4] FIG. 4 is a flow chart showing the flow of movement processing in the automatic trimming device 10 according to the present invention. [Diagram 5] FIG. 2 is a flowchart showing the flow of prohibited object elimination processing in the automatic trimming device 10 according to the present invention. [Figure 6] FIG. 2 is a flow chart showing the flow of adjustment processing in the automatic trimming device 10 according to the present invention. [Figure 7] FIG. 2 is an explanatory diagram showing an example of image data handled in the automatic trimming device 10 according to the present invention. [Figure 8] 1A to 1C are explanatory diagrams showing an example of specifying an object for image data and setting various frames in the automatic trimming device 10 according to the present invention. [Figure 9] 2 is an explanatory diagram showing an example of the positional relationship between image data and a trimming frame after automatic trimming processing in the automatic trimming device 10 according to the present invention. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an example of an automatic trimming device according to an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a block diagram showing the configuration of an automatic trimming device 10 according to the present invention. As shown in Fig. 1, the automatic trimming device 10 includes at least an image data acquisition unit 11, a detection unit 12, a specification unit 13, a frame setting unit 14, a zoom unit 15, a movement unit 16, an adjustment unit 17, a trimming unit 18, and a storage unit 19.

[0016] The automatic trimming device 10 may be a device designed as a dedicated machine, but can be realized by a general computer. In this case, the automatic trimming device 10 is equipped with a CPU (Central Processing Unit), memory, hard disk drive, and other storage devices that would normally be equipped in a general computer (not shown). In addition, a GPU (Graphics Processing Unit: image processing device) may be included as necessary. Needless to say, various processes are executed by programs to cause such a general computer to function as the automatic trimming device 10 of this example.

[0017] In addition, the automatic trimming device 10 may function as a stand-alone device independent of a communication network, or some or all of the functions of the automatic trimming device 10 may be provided in a server device, and the automatic trimming device 10 may be used by connecting to a terminal device via a communication network.

[0018] The image data acquisition unit 11 has a function of acquiring target image data. In this example, the acquired image data is mainly image data obtained by a photographer such as a cameraman taking a photograph with a specific shooting intention.

[0019] The detection unit 12 has a function of detecting a predetermined object appearing in the image data by image recognition processing. Here, the predetermined object refers to various objects that may be included in the image data, such as people, animals, objects such as vases, furniture, and accessories such as things owned by people. Image recognition is performed by setting in advance what kind of object is required to be recognized. The image recognition processing may be performed by any method as long as it can appropriately detect the object. For example, it is possible to perform the image recognition processing based on a neural network (trained model) that has been trained in advance by deep learning to recognize the predetermined object. The detection unit 12 specifies the detection range of the object by dividing the area occupied by the detected object by a predetermined method, for example, by dividing it by a rectangle. In addition, a target object may be detected on a pixel-by-pixel basis using segmentation AI.

[0020] The specification unit 13 has a function of specifying a required object that must be included within the template frame among the detected objects. Here, a required object refers to an object that is preset to be included within the template frame when included in image data. The required object may be set in any way, but for example, a person and accessories of the person such as a bag, hat, hair accessory, and clothing that the person carries may be set as the required object.

[0021] The specification unit 13 also has a function of specifying an allowable object that is allowed to fit within the template frame. Here, an allowable object refers to an object that is allowed to fit within the template frame when included in image data, but is not required to be included within the template frame. The allowable object may be set in any way, but examples of the allowable object include a large vase placed next to a person, an animal that appears together with a person, and the like.

[0022] The specification unit 13 also has a function of specifying prohibited objects that should not be placed within the template frame. Here, a prohibited object refers to an object that is prohibited from being placed within the template frame when included in image data. Prohibited objects may be set in any way, but examples of the prohibited objects include unintended objects that occur during photography in a photo studio, such as a panel bar, a piece of backing paper, a piece of a sliding door, a floor in front of a sliding door, a roll of paper, or a staff member.

[0023] Although the detection unit 12 and the identification unit 13 have been described with separate functions, they may be configured to perform these as a series of processes. For example, in a configuration in which processing is performed based on a trained model that has previously been trained to detect an object and identify the attributes of the object, the trained model serves as both the detection unit 12 and the identification unit 13, and such a configuration may be used.

[0024] The frame setting unit 14 has a function of setting a template frame for image data based on a predetermined rule. Here, the template frame refers to a frame of a predetermined size that is set in advance as a size for trimming image data. The template frame may be selected by a user who desires trimming, or may be automatically selected based on the size of the image data. The relationship between the size of the template frame and the size of the image indicated by the image data is not particularly limited, but it is preferable to select a template frame smaller than the size of the image data, since the use of a template frame larger than the size of the image data necessarily requires enlargement processing and causes deterioration of image quality. The predetermined rule for setting the template frame may be any rule, but for example, a rule for setting the initial position of the template frame so that the center coordinates of the image indicated by the image data and the center coordinates of the template frame overlap can be considered. Alternatively, a rule for determining the center coordinates of a required object, which is a subject, and setting the initial position of the template frame so that the center coordinates overlap the center coordinates of the template frame can be considered.

[0025] The scaling unit 15 has a function of executing a scaling process on the image data so that the required objects fit within the template frame. The scaling unit 15 executes the scaling process or the reduction process until all areas identified as required objects fit completely within the template frame. Here, the scaling process is executed with a predetermined reference point as the center, and for example, the scaling process is executed with the central coordinates of the template frame as the reference.

[0026] Furthermore, the scaling unit 15 may further perform padding processing. Here, padding processing refers to performing a scaling process so as to create a predetermined interval (gap) between the template frame and the area specified as the required object. The predetermined interval may be any setting as long as it is set in advance, and may be, for example, a setting that specifies the interval by the number of pixels.

[0027] In addition, the enlargement / reduction processing in the enlargement / reduction unit 15 may be limited in terms of the enlargement / reduction ratio. It is possible to set the enlargement / reduction ratio in order to protect the photographer's intention, on the premise that an extremely large enlargement / reduction ratio is likely to result in trimming that is far from the photographer's intention. Specifically, when the original size of the original image data is set to a enlargement / reduction ratio of 100%, it is possible to limit the enlargement / reduction processing to a range of 92%<enlargement / reduction ratio<110%. Note that this limit on the enlargement / reduction processing may be determined for each process in the enlargement / reduction unit 15, or may be determined in the adjustment unit 17 described later. In addition, when the enlargement / reduction ratio limit is exceeded, the previous state may be maintained, or the initial state may be restored.

[0028] The enlargement / reduction unit 15 may have a function of setting a judgment frame that is larger than the size of the template frame at the initial position initially set in the image data by a predetermined size, and executing a enlargement / reduction process to fit the permissible object within the template frame when the permissible object is within the judgment frame. Here, the judgment frame refers to a frame for determining whether or not the photographer intends to photograph the permissible object included in the image data together with the mandatory object. When a template frame smaller than the size of the image data is selected, the judgment frame is larger than the size of the template frame by a predetermined size and smaller than the image data by a predetermined size. That is, the following holds: template frame<judgment frame<image data. As a more detailed example, it is possible to set the position of the judgment frame at a position that is 1 / 3 of the distance from the image data side when the distance between the template frame and the image data is divided into three. The enlargement / reduction process is executed to fit the mandatory object within the template frame, but at that time, a permissible object that does not fit within the template frame may occur. If such a permissible object is intended by the photographer to be photographed together with the mandatory object, it is necessary to adjust it so that it fits within the template frame. Therefore, a judgment frame is set, and for permissible objects that fit within the judgment frame, it is presumed that the photographer intended to photograph them together with the required objects, and the object is fitted into the template frame by enlargement / reduction processing. On the other hand, for permissible objects that do not fit within the judgment frame, since in many cases a part of the subject also protrudes from the image data, it is presumed that the photographer also took the photograph under the assumption that a part of the subject would protrude, and processing to fit the object into the template frame is not performed.

[0029] The scaling unit 15 may also have a function of executing a scaling process so that a prohibited object does not remain within the template frame within a range that does not meet a predetermined prohibition condition. If a prohibited object identified by the identification unit 13 remains within the template frame, the prohibited object is to be removed from the template frame, and therefore the scaling unit 15 further executes a scaling process to remove the prohibited object from the template frame. The prohibition condition refers to a condition that must not be violated when removing a prohibited object, such as not exceeding a limited range of the scaling ratio.

[0030] The moving unit 16 has a function of executing a moving process of the image data relative to the template frame based on a predetermined criterion including a criterion of the amount of top, bottom, left and right margins for the required object in the template frame. Here, the moving process refers to a process of adjusting the position of the image data relative to the template frame by translation without changing the size of the image data. The predetermined criterion in the moving process refers to a criterion for determining that the moving process is necessary. Examples of the predetermined criterion include a margin amount criterion such as a minimum number of pixels of margin required, a criterion regarding the ratio of top and bottom margins such that the upper margin must be larger so that the ratio of the lower margin to the upper margin is at least 1:1.5, and a criterion that the center of the subject should be located as much as possible in the center of the template frame within the limit of the left and right movement amount. In addition, regarding the top and bottom margins, it is also necessary to determine whether the photo is a full-body photo. This is because image data in which a part of the subject protrudes outside the shooting area is not a full-body photo, and a process of providing margins is not required for the part where the part of the subject protrudes. The subject center may be calculated using only the required objects, or may be calculated using the overall subject center including the required objects and the permitted objects as a reference, treating the permitted objects determined to be within the template frame in the same manner. The limit on the amount of movement to the left and right is set to protect the photographer's intention, and may be a fixed limit of a certain number of pixels beyond which the object cannot be moved, or may be a limit set by setting a number of pixels that the object can be moved based on the size of the image data. When the limit on the amount of movement is exceeded, the previous state may be maintained, or the initial state may be restored. The order of processing by the zoom unit 15 and the movement unit 16 may be reversed.

[0031] The adjustment unit 17 has a function of finely adjusting the relationship between the image data and the template frame. Although the required object should be placed in an appropriate position within the template frame through the processing in the enlargement / reduction unit 15 and the movement unit 16, a situation may arise in which fine adjustment is required in detail. For example, image clipping may occur at the edge of the template frame. The adjustment unit 17 executes a process of adjusting the target of fine adjustment, such as image clipping. The adjustment unit 17 may also execute a process of checking whether various restrictions, such as restrictions on the enlargement / reduction ratio and the movement amount, are observed.

[0032] The trimming unit 18 has a function of outputting the inner area of ​​the template frame from the image data in the determined positional relationship as a trimmed image. The processing in the trimming unit 18 may be configured to cut the image data with the template frame and output the trimmed image, or may be configured to output information on the enlargement / reduction ratio and movement amount of the image data with respect to the template frame as information for trimming.

[0033] The memory unit 19 has the function of storing data required for various processes performed in the automatic trimming device 10, which includes the image data acquisition unit 11, the detection unit 12, the identification unit 13, the frame setting unit 14, the zoom unit 15, the movement unit 16, the adjustment unit 17, the trimming unit 18, etc., as well as data obtained as a result of the processes.

[0034] Next, the flow of the automatic trimming process in the automatic trimming device 10 will be described. FIG. 2 is a flow chart showing the flow of the automatic trimming process in the automatic trimming device 10 according to the present invention. As shown in FIG. 2, the automatic trimming process is started by acquiring image data in the automatic trimming device 10 (step S101). Next, the automatic trimming device 10 detects a predetermined object from the acquired image data (step S102). Next, the automatic trimming device 10 specifies a required object, an allowed object, a prohibited object, etc. from the detected objects (step S103). Next, the automatic trimming device 10 sets a template frame for the image data (step S104). Next, the automatic trimming device 10 performs a zoom process so that the required object (as well as the allowed object if the conditions are met) fits within the template frame (step S105). Next, the automatic trimming device 10 performs a movement process so that the required object (as well as the allowed object if the conditions are met) is appropriately positioned within the template frame (step S106). Next, if a prohibited object is present within the template frame, the automatic trimming device 10 executes a prohibited object elimination process (step S107). Then, the automatic trimming device 10 executes an adjustment process (step S108). Finally, the automatic trimming device 10 executes a trimming process based on the relationship between the determined template frame and the image data (step S109), and ends the automatic trimming process. Note that the order of execution of the enlargement / reduction process (step S105) and the movement process (step S106) may be reversed.

[0035] Next, the flow of the enlargement / reduction process in the automatic trimming device 10 will be described. FIG. 3 is a flow chart showing the flow of the enlargement / reduction process in the automatic trimming device 10 according to the present invention. In a situation where the required objects, allowed objects, prohibited objects, etc. are specified and the template frame is set to the initial position for the image data, as shown in FIG. 3, the enlargement / reduction process is started by the automatic trimming device 10 enlarging / reducing the image data so that the required objects are included in the template frame (step S201). Next, the automatic trimming device 10 performs padding processing so that a predetermined interval is secured between the required objects and the template frame (step S202). Next, the automatic trimming device 10 judges whether or not the allowed object is present in the image data (step S203). If the allowed object is not present in the image data (S203-N), the automatic trimming device 10 ends the enlargement / reduction process. If the allowed object is present in the image data (S203-Y), the automatic trimming device 10 sets a judgment frame (where, judgment frame<image data) that is a predetermined size larger than the size of the initial arrangement of the template frame for the image data (step S204). Then, the automatic trimming device 10 judges whether the permissible object fits within the judgment frame (step S205). If the permissible object does not fit within the judgment frame (S205-N), the automatic trimming device 10 ends the scaling process. If the permissible object fits within the judgment frame (S205-Y), the automatic trimming device 10 performs scaling so that the permissible object fits within the template frame (step S206), and ends the scaling process.

[0036] Next, the flow of the movement process in the automatic trimming device 10 will be described. FIG. 4 is a flow chart showing the flow of the movement process in the automatic trimming device 10 according to the present invention. As shown in FIG. 4, the movement process is started by the automatic trimming device 10 by determining whether the image data is a full-body photograph of the subject (step S301). If the image data is not a full-body photograph (S301-N), the automatic trimming device 10 proceeds to step S304, which will be described later. If the image data is a full-body photograph (S301-Y), the automatic trimming device 10 determines whether there is only a predetermined value or less between the template frame above and below the mandatory object (step S302). If there is a space larger than the predetermined value between the mandatory object template frame (S301-N), the automatic trimming device 10 proceeds to step S304, which will be described later. If there is only a predetermined value or less between the mandatory object template frame and the mandatory object (S301-Y), vertical movement is performed so that there is a predetermined value or more between the mandatory object and the template frame above and below the mandatory object (step S303). Finally, the subject center of the required object is determined, and horizontal movement is performed so that the subject center approaches the center of the template frame (step S304), and the movement process ends.

[0037] Next, the flow of the prohibited object elimination process in the automatic trimming device 10 will be described. FIG. 5 is a flow chart showing the flow of the prohibited object elimination process in the automatic trimming device 10 according to the present invention. As shown in FIG. 5, the prohibited object elimination process is started by the automatic trimming device 10 by determining whether or not a prohibited object exists within the template frame (step S401). If a prohibited object does not exist within the template frame (S401-N), the automatic trimming device 10 ends the prohibited object elimination process. If a prohibited object exists within the template frame (S401-Y), the automatic trimming device 10 executes an enlargement process to push the prohibited object (or the prohibited object plus a margin) out of the template frame (step S402), and ends the prohibited object elimination process.

[0038] Next, the flow of the adjustment process in the automatic trimming device 10 will be described. FIG. 6 is a flow chart showing the flow of the adjustment process in the automatic trimming device 10 according to the present invention. As shown in FIG. 6, the adjustment process is started by determining whether or not a mask break has occurred in the automatic trimming device 10 (step S501). If a mask break has not occurred (S501-N), the automatic trimming device 10 proceeds to step S304, which will be described later. If a mask break has occurred (S501-Y), the automatic trimming device 10 executes an enlargement process to fill in the missing area (step S502). Next, the automatic trimming device 10 determines whether or not the enlargement / reduction ratio and the movement amount are within the range of the limit value (step S503). If the enlargement / reduction ratio or the movement amount exceeds the range of the limit value (S503-N), the automatic trimming device 10 stops the entire trimming process and resets everything to the initial value (step S504), and then proceeds to step S505. If the enlargement / reduction ratio and the movement amount are within the range of the limit value (S503-Y), the automatic trimming device 10 maintains the enlargement / reduction ratio and the movement amount information at that time and proceeds to step S505. Finally, the automatic trimming device 10 outputs the trimming information (step S505) and ends the adjustment process.

[0039] Next, the automatic trimming process in the automatic trimming device 10 will be described based on a concrete example. Fig. 7 is an explanatory diagram showing an example of image data handled by the automatic trimming device 10 according to the present invention. The image data shown in Fig. 7 includes a person and a flower as examples of objects, and also shows a state in which the background paper has run out at the lower left.

[0040] FIG. 8 is an explanatory diagram showing an example of specifying objects for image data and setting various frames in the automatic trimming device 10 according to the present invention. When the image recognition process is performed on the image data shown in FIG. 7 to specify objects, for example, as shown in FIG. 8, a "person" as a required object, a "flower" as an allowed object, and a "piece of paper backing" as a prohibited object are specified, and the specified areas are managed as rectangular areas. Also, as shown in FIG. 8, a template frame arranged at an initial position and a judgment frame larger than the template frame are set for the image data. In the example shown in FIG. 8, the "flower" as an allowed object protrudes from the template frame arranged at the initial position, but is within the range of the judgment frame, so that it is determined to be an allowed object to be contained inside the template frame. In the example shown in FIG. 8, a scaling process is performed to fit the "flower" as an allowed object inside the template frame, and allowed objects such as the "flower" are also included in the calculation of the subject center, so that both the "person" and the "flower" are recognized as subjects, and a moving process is performed so that the subject center is set to the center of the entire "person" and the "flower". When the permissible objects are not included in the calculation of the subject center, the subject center is calculated based only on the "person" and the movement process is executed so that the subject center is located at the center of the whole image.

[0041] Fig. 9 is an explanatory diagram showing an example of the positional relationship between image data and a trimming frame after automatic trimming processing in the automatic trimming device 10 according to the present invention. The result of performing enlargement / reduction processing and movement processing on the image data shown in Fig. 8 is the relationship between the template frame and image data shown in Fig. 9. Fig. 9 shows that the trimming information obtained for the image data with respect to the template frame in the initial position is a scaling ratio of 0.97 (reduced to 97%), a horizontal movement amount of -20 (moved 20 pixels to the left), and a vertical movement amount of 5 (moved 5 pixels upward). Note that, as shown in Fig. 9, no mask cut has occurred, and since the prohibited object "cut back paper" is outside the template frame and does not pose a problem, fine adjustment is not necessary.

[0042] As described above, the automatic trimming device 10 of the present invention acquires target image data, detects specified objects appearing in the image data by image recognition processing, identifies required objects that must be contained within the template frame from among the detected objects, sets a template frame for the image data based on specified rules, performs scaling processing on the image data so that the required objects fit within the template frame, performs processing to move the image data relative to the template frame based on specified criteria including criteria for the amount of margin above, below, left, and right for the required objects within the template frame, and outputs the area inside the template frame from the image data in the determined positional relationship as a trimming image, so that trimming that includes all required objects and has appropriate margins set can be automatically performed.

[0043] In addition, since permissible objects are also permitted to be contained within the frame within a range that does not fall under predetermined prohibition conditions, permissible objects existing within the template frame can be recognized as subjects included based on the photographer's intention, and can be handled by containing them within the frame while respecting the photographer's intention.

[0044] In addition, a judgment frame is set that is a predetermined size larger than the size of the template frame at the initial position initially set in the image data, and if an allowed object is within the judgment frame, a scaling process is performed to fit the allowed object within the template frame, so that an allowed object that is presumed to have been intended by the photographer to be included in the image data can be fitted within the template frame, making it possible to perform trimming that respects the photographer's intentions. Conversely, an allowed object that protrudes beyond the judgment frame is presumed to be an allowed object that the photographer did not intend to fit in its entirety in the image data, and such an allowed object is not included in the template frame. This can also be said to be a trimming process that reflects the photographer's intentions.

[0045] Furthermore, prohibited objects that should not be contained within the template frame are specified, and enlargement / reduction processing is performed so that the prohibited objects do not remain within the template frame within a range that does not meet predetermined prohibition conditions, making it possible to perform automatic trimming so that targets set as prohibited objects do not remain within the template frame. [Explanation of symbols]

[0046] 10 Automatic trimming device 11 Image data acquisition section 12 Detection section 13 Specific section 14 Frame setting section 15 Scaling section 16 Mobile section 17 Adjustment section 18 Trimming Section 19 Memory section

Claims

1. An automatic cropping program for causing a computer to perform a process of automatically cropping image data acquired by photographing an object so that the object fits within a frame of a predetermined size, The computer includes: an image data acquisition function for acquiring target image data; A detection function of detecting a predetermined object that is preset as a recognition target in image recognition by executing an image recognition process on the image data; A specification function for specifying a required object that is preset as an object that must be included within the frame among the detected objects; a frame setting function for setting an initial position of the frame with respect to the image data based on a predetermined rule for setting an initial position of the frame with respect to the image data; a scaling function for executing a scaling process on the image data so that the required object fits within the frame; a movement function that executes a movement process of the image data based on a predetermined criterion including criteria for the amount of blank space above, below, left, and right for the required object within the frame, so that the positional relationship of the image data with respect to the frame satisfies the predetermined criterion; a trimming function for outputting an inner area of ​​the frame from the image data as a trimmed image in the determined positional relationship between the frame and the image data; The specifying function further includes a function of specifying an allowable object that is different from the required object and is preset as an object that is allowed to fit within the frame, The enlargement / reduction function sets an enlargement determination frame that is a predetermined size larger than the size of the frame at the initial position initially set in the image data, and when the permissible object protrudes from the frame but is contained inside the enlargement determination frame, performs a reduction process on the image data so as to contain the permissible object within the frame. Auto-cropping program.

2. An automatic cropping program for causing a computer to perform a process of automatically cropping image data acquired by photographing an object so that the object fits within a frame of a predetermined size, The computer includes: an image data acquisition function for acquiring target image data; A detection function of detecting a predetermined object that is preset as a recognition target in image recognition by executing an image recognition process on the image data; A specification function for specifying a required object that is preset as an object that must be included within the frame among the detected objects; a frame setting function for setting an initial position of the frame with respect to the image data based on a predetermined rule for setting an initial position of the frame with respect to the image data; a scaling function for executing a scaling process on the image data so that the required object fits within the frame; a movement function that executes a movement process of the image data based on a predetermined criterion including criteria for the amount of blank space above, below, left, and right for the required object within the frame, so that the positional relationship of the image data with respect to the frame satisfies the predetermined criterion; a trimming function for outputting an inner area of ​​the frame from the image data as a trimmed image in the determined positional relationship between the frame and the image data; The identification function further includes a function of identifying a prohibited object that is preset as an object that should not be placed within the frame, The enlargement / reduction function executes a process of enlarging or reducing the image data so that the prohibited object is removed from the frame within a range of a predetermined enlargement / reduction ratio limit as a condition for the process of removing the prohibited object. Auto-cropping program.

3. An automatic cropping device for automatically cropping image data acquired by photographing an object so that the object fits within a frame of a predetermined size, An acquisition unit for acquiring target image data; a detection unit that detects a predetermined object that is set in advance as a recognition target in image recognition by executing an image recognition process on the image data; an identification unit that identifies a required object that is preset as an object that must be included within the frame from among the detected objects; a frame setting unit that sets an initial position of the frame with respect to the image data based on a predetermined rule for setting an initial position of the frame with respect to the image data; a scaling unit that performs a scaling process on the image data so that the required object fits within the frame; a moving unit that executes a process of moving the image data based on a predetermined criterion including criteria for the amount of blank space above, below, left, and right for the required object within the frame so that a positional relationship of the image data with respect to the frame satisfies the predetermined criterion; a trimming unit that outputs an inner area of ​​the frame from the image data as a trimmed image in the determined positional relationship between the frame and the image data, the specifying unit further includes a function of specifying an allowable object that is different from the required object and is preset as an object that is allowed to fit within the frame, The enlargement / reduction unit sets an enlargement determination frame that is a predetermined size larger than the size of the frame at the initial position initially set in the image data, and when the permissible object protrudes from the frame but is contained inside the enlargement determination frame, performs a reduction process on the image data so as to contain the permissible object within the frame. Automatic trimming device.

4. An automatic cropping device for automatically cropping image data acquired by photographing an object so that the object fits within a frame of a predetermined size, An acquisition unit for acquiring target image data; a detection unit that detects a predetermined object that is set in advance as a recognition target in image recognition by executing an image recognition process on the image data; an identification unit that identifies a required object that is preset as an object that must be included within the frame from among the detected objects; a frame setting unit that sets an initial position of the frame with respect to the image data based on a predetermined rule for setting an initial position of the frame with respect to the image data; a scaling unit that performs a scaling process on the image data so that the required object fits within the frame; a moving unit that executes a process of moving the image data based on a predetermined criterion including criteria for the amount of blank space above, below, left, and right for the required object within the frame so that a positional relationship of the image data with respect to the frame satisfies the predetermined criterion; a trimming unit that outputs an inner area of ​​the frame from the image data as a trimmed image in the determined positional relationship between the frame and the image data, The identification unit further includes a function of identifying a prohibited object that is preset as an object that should not be placed within the frame, The enlargement / reduction unit performs enlargement / reduction processing of the image data so that the prohibited object is removed from the frame within a range of a predetermined enlargement / reduction ratio limit as a condition for processing to remove the prohibited object. Automatic trimming device.

5. An automatic cropping method for automatically cropping image data acquired by photographing an object so that the object fits within a frame of a predetermined size, the method comprising the steps of: An acquisition procedure for acquiring target image data; a detection step of detecting a predetermined object, which is preset as a recognition target in image recognition, by executing an image recognition process on the image data; a step of identifying a required object that is preset as an object that must be included within the frame from among the detected objects; a frame setting step of setting an initial position of the frame with respect to the image data based on a predetermined rule for setting an initial position of the frame with respect to the image data; a scaling step of scaling the image data so that the required object fits within the frame; a moving procedure for executing a process of moving the image data based on a predetermined criterion including criteria for the amount of blank space above, below, left, and right for the required object within the frame, so that a positional relationship of the image data with respect to the frame satisfies the predetermined criterion; a cropping step of outputting an inner area of ​​the frame from the image data as a cropped image in the determined positional relationship between the frame and the image data, The specifying step further includes a function of specifying an allowable object that is different from the required object and is preset as an object that is allowed to fit within the frame, The enlargement / reduction step sets an enlargement determination frame that is a predetermined size larger than the size of the frame at the initial position initially set in the image data, and when the permissible object protrudes from the frame but is contained inside the enlargement determination frame, performs a reduction process on the image data so as to contain the permissible object within the frame. Auto crop method.

6. An automatic cropping method for automatically cropping image data acquired by photographing an object so that the object fits within a frame of a predetermined size, the method comprising the steps of: An acquisition procedure for acquiring target image data; a detection step of detecting a predetermined object, which is preset as a recognition target in image recognition, by executing an image recognition process on the image data; a step of identifying a required object that is preset as an object that must be included within the frame from among the detected objects; a frame setting step of setting an initial position of the frame with respect to the image data based on a predetermined rule for setting an initial position of the frame with respect to the image data; a scaling step of scaling the image data so that the required object fits within the frame; a moving procedure for executing a process of moving the image data based on a predetermined criterion including criteria for the amount of blank space above, below, left, and right for the required object within the frame, so that a positional relationship of the image data with respect to the frame satisfies the predetermined criterion; a cropping step of outputting an inner area of ​​the frame from the image data as a cropped image in the determined positional relationship between the frame and the image data, The identification step further includes a function of identifying a prohibited object that is preset as an object that should not be placed within the frame, The enlargement / reduction step enlarges / reduces the image data so that the prohibited object is removed from the frame within a range of a predetermined enlargement / reduction ratio limit as a condition for the process of removing the prohibited object. Auto crop method.

Images