Image processing method, recording medium, and image processing system

Abstract

The present application provides an image processing method, program and image processing system capable of simply enjoying Chinese characters. The image processing method includes the following steps: an image analysis step (S14) of analyzing an acquired image and extracting an object from the image; a Chinese character string determination step (S16) of determining a Chinese character string including one or more Chinese characters related to the object; an alphabet string determination step (S16) of determining an alphabet string related to the meaning of the Chinese character string; a layout determination step (S18) of determining a layout of synthesizing the Chinese character string and the alphabet string in the image; and a synthesis step (S20) of generating a synthesized image by synthesizing the Chinese character string and the alphabet string in the image according to the layout.

Description

Technical Field

[0001] This invention relates to an image processing method, a recording medium, and an image processing system, and more particularly to a project (contents) for appreciating Chinese characters. Background Technology

[0002] For foreign visitors to Japan, especially those who don't usually use kanji, kanji is a rare sight. Kanji is appreciated and incorporated into items such as clothing and postcards. Furthermore, electronic products featuring kanji are popular and purchased by foreigners.

[0003] As a way to help foreigners appreciate Chinese characters, examples include making seals that use Japanese kanji and letters on the seal face, and creating kanji characters for personal names by arranging one or more borrowed characters in a way that corresponds to the pronunciation of foreign names. However, these are all just translating the original text into kanji, or the kanji shown to foreigners are simply what Japanese people individually perceive as kanji.

[0004] Patent Document 1 describes an information display system that overlays character information related to the subject of a photographed image onto the photographed image. This system overlays the name of a "mountain" onto a photographed image of a "mountain" and the name of a "lake" onto a photographed image of a "lake".

[0005] Patent Document 2 describes a camera capable of searching for images recorded in its recording unit. The camera described in this document searches for images recorded in its recording unit using keywords input via an electronic keypad displayed on a liquid crystal display.

[0006] Patent document 3 describes a home theater system for replaying movies, etc. In this document, as an explanation of PG streaming, it describes a graphics stream containing movie subtitles and other graphics that should be closely synchronized with the images; this graphics stream is a stream for multiple languages, including English. Furthermore, PG is an abbreviation for Presentation Graphics.

[0007] Patent Document 1: Japanese Patent Application Publication No. 2018-151829

[0008] Patent Document 2: Japanese Patent No. 6063536

[0009] Patent Document 3: Japanese Patent No. 5341946

[0010] However, there are currently no projects that allow foreign tourists visiting Japan to easily appreciate kanji. Furthermore, none of the inventions described in Patent Documents 1 to 3 are inventions that allow foreign tourists to easily appreciate kanji. Summary of the Invention

[0011] The present invention was made in view of this situation, and its purpose is to provide an image processing method, program and image processing system that enables easy appreciation of Chinese characters.

[0012] To achieve the above objectives, the following inventive method is provided.

[0013] The image processing method involved in Method 1 is as follows: an image analysis step, which analyzes the acquired image and extracts the object from the image; a Chinese character string determination step, which determines a Chinese character string containing one or more Chinese characters related to the object; a letter string determination step, which determines a letter string related to the meaning of the Chinese character string; a layout determination step, which determines the layout for combining the Chinese character string and the letter string in the image; and a synthesis step, which synthesizes the Chinese character string and the letter string in the image according to the layout to generate a synthesized image.

[0014] According to the first method, Chinese character strings related to the objects contained in the acquired image and letter strings related to the meaning of the Chinese character strings are combined in the image. Therefore, the content of the Chinese character string can be understood based on the meaning of the letter strings, allowing foreigners and others to easily appreciate Chinese characters.

[0015] The objects contained in the image can be applied to subjects in images such as people, natural structures, and buildings. Objects can also be applied to backgrounds in images such as the sky and landscapes. The image analysis process can extract one object from multiple objects, or it can extract two or more objects.

[0016] Kanji can be used in Japanese. Kanji can also be used in foreign countries such as China.

[0017] Examples of Chinese character strings related to objects include strings representing the content of an object, strings representing the image of an object, and strings that can be imagined by the object.

[0018] Alphabetic strings can be used to represent English words and other foreign language words, including Chinese characters. Alphabetic strings can contain specific symbols such as hash tags.

[0019] Examples of letter strings related to Chinese character strings include letter strings that represent the meaning of Chinese character strings, letter strings that represent the image of Chinese character strings, and letter strings that can be imagined from Chinese character strings.

[0020] A layout can contain information about the position of objects. A layout can contain information such as the position, color, and font of Chinese characters in an image. A layout can also contain information such as the position, color, and font of alphabetic strings in an image.

[0021] The location information of the object, the location information of the Chinese character string, and the location information of the letter string can be applied to the coordinate values ​​in the image.

[0022] The second method can be structured as follows: In the image processing method of the first method, the layout determination process adopts the layout of configuring letter strings in the area where Chinese character strings are configured.

[0023] According to the second method, it is possible to comprehensively recognize Chinese character strings and letter strings visually. Furthermore, it is possible to represent the unity of Chinese character strings and letter strings.

[0024] The third approach can be structured as follows: In the image processing method of the first or second approach, the layout determination step adopts a layout in which at least a portion of the Chinese character string is placed at a position that overlaps with the object.

[0025] According to the third method, it is possible to create images with excellent design and composite images containing Chinese character strings and letter strings. Furthermore, it is possible to grasp the correspondence between objects and Chinese character strings.

[0026] The fourth method can be structured as follows: In any of the image processing methods from the first to the third method, the layout determination process adopts a layout that places the Chinese character string in the foreground of the object.

[0027] According to method 4, Chinese character strings can be made prominent in the synthesized image.

[0028] In the fourth method, a layout that places the letter string in the foreground of the object can be used.

[0029] The fifth method can be structured as follows: In any of the image processing methods from the first to the fourth method, the layout determination step adopts a layout that places the object in the foreground of the Chinese character string.

[0030] According to method 5, one can appreciate the Chinese characters related to the object without compromising the presence of the object in the synthesized image.

[0031] In the fifth method, a layout can be used to place the object in the foreground of the letter string.

[0032] The sixth method can be structured as follows: In any of the image processing methods from the first to the fifth method, the layout determination process adopts a layout that allows the object to pass through at least a portion of the Chinese character string.

[0033] According to method 6, the Chinese character string can be made prominent in the composite image without compromising the presence of the object in the composite image, and the Chinese characters related to the object can be appreciated.

[0034] In the sixth method, it is possible to use a layout that allows the object to be transparent for at least a portion of the letter string.

[0035] The seventh method can be structured as follows: In any of the image processing methods from the first to the sixth method, the image analysis step extracts the colors used in the object, and the layout determination step uses the colors extracted as the colors of the Chinese character string.

[0036] According to method 7, it is possible to achieve color coordination between the image and the Chinese character string in the synthesized image.

[0037] In method 7, the colors of the extracted image can be used as the colors of the letter strings. This allows for color harmony in the overall synthesized image.

[0038] Method 8 can be structured as follows: In any of the image processing methods from Method 1 to Method 7, the image analysis process extracts multiple objects from the image, and the layout determination process adopts a layout in which Chinese character strings are configured in each of the multiple objects.

[0039] According to method 8, it is possible to generate composite images that emphasize Chinese character strings and have excellent design.

[0040] In method 8, it is possible to combine a Chinese character string and a letter string as a group for an object.

[0041] The ninth method can be structured as follows: In any of the image processing methods from the first to the eighth method, the layout determination process adopts a layout that configures the Chinese character string to an orientation corresponding to the orientation of the object.

[0042] According to method 9, it is possible to generate composite images with excellent design that demonstrate the unity of the object and the Chinese character string.

[0043] In method 9, the orientation of the letter string can be made to correspond to the orientation of the object.

[0044] The 10th method can be structured as follows: In any of the image processing methods from the 1st to the 9th method, the layout determination step adopts a layout in which one or more Chinese characters contained in the Chinese character string are placed at the edge of the image.

[0045] According to method 10, it is possible to generate composite images that emphasize Chinese characters and have excellent design.

[0046] Method 11 can be structured as follows: In any of the image processing methods from Method 1 to Method 10, the Chinese character string determination step determines one or more Chinese character strings of the synthesized object from multiple candidate Chinese character strings related to the object.

[0047] According to method 11, it is possible to expand the selection options for the extracted Chinese character string.

[0048] Method 12 can be structured as follows: In any of the image processing methods from Method 1 to Method 11, the letter string determination step determines the letter string of the synthesized object from multiple candidate letter strings related to the meaning of the Chinese character string.

[0049] According to method 12, it is possible to select one or more letter strings from multiple candidate letter strings for a Chinese character string.

[0050] The 13th method can be structured as follows: In any of the image processing methods from the 1st to the 12th methods, a Chinese character string modification step is included, which modifies the Chinese character string determined in the Chinese character string determination step into another Chinese character string related to the object.

[0051] According to method 13, it is possible to modify Chinese character strings.

[0052] In method 13, a user interface can be used to allow users to manually modify Chinese character strings.

[0053] In method 13, the letter string can be manually or automatically modified based on the modified Chinese character string.

[0054] Method 14 can be structured as follows: In any of the image processing methods from Method 1 to Method 13, a letter string modification step is included, which modifies the letter string determined in the letter string determination step into another letter string related to the meaning of the Chinese character string.

[0055] According to method 14, it is possible to modify the letter string.

[0056] In method 14, a user interface is available, allowing users to manually modify letter strings.

[0057] The 15th method can be structured as follows: In any of the image processing methods from the 1st to the 14th methods, the method includes: a printing process, which uses a printing device to print a composite image.

[0058] According to method 15, it is possible to provide printed materials containing composite images. Thus, it is possible to appreciate Chinese characters using printed materials.

[0059] Method 16 involves a program that enables a computer to perform the following functions: an image analysis function that analyzes the acquired image and extracts objects from the image; a Chinese character string determination function that determines a Chinese character string containing one or more Chinese characters related to the object; an alphabetic string determination function that determines an alphabetic string related to the meaning of the Chinese character string; a layout determination function that determines the layout when combining the Chinese character string and the alphabetic string in the image; and a synthesis function that synthesizes the Chinese character string and the alphabetic string in the image according to the layout to generate a synthesized image.

[0060] According to method 16, the same effect as method 1 can be obtained.

[0061] In the 16th method, it is possible to appropriately combine the same items as those specified in methods 2 through 15. In this case, the constituent elements responsible for a specific process or function in the image processing method can be grasped as the constituent elements of the program responsible for the corresponding process or function.

[0062] The image processing system involved in Method 17 is an image processing system comprising: an image analysis unit that analyzes the acquired image and extracts objects from the image; a Chinese character string determination unit that determines a Chinese character string containing one or more Chinese characters related to the object; an alphabetic string determination unit that determines an alphabetic string related to the meaning of the Chinese character string; a layout determination unit that determines the layout when combining the Chinese character string and the alphabetic string in the image; and a synthesis unit that synthesizes the Chinese character string and the alphabetic string in the image according to the layout to generate a synthesized image.

[0063] According to method 17, the same effect as method 1 can be obtained.

[0064] In the 17th method, the same items as those specified in the 2nd to 15th methods can be appropriately combined. In this case, the constituent elements responsible for a specific process or function in the image processing method can be grasped as the constituent elements of the image processing system responsible for the corresponding process or function.

[0065] The 18th method can be configured as follows: In the image processing system of the 17th method, there is a printing unit for printing composite images.

[0066] According to method 18, it is possible to provide printed materials containing composite images. Thus, it is possible to appreciate Chinese characters using printed materials.

[0067] The image processing system involved in Method 19 is an image processing system that is connected to a network and is capable of communication. It includes: an image analysis unit that analyzes the acquired image and extracts objects from the image; a Chinese character string determination unit that determines a Chinese character string containing one or more Chinese characters related to the object; an alphabetic string determination unit that determines an alphabetic string related to the meaning of the Chinese character string; a layout determination unit that determines the layout when combining the Chinese character string and the alphabetic string in the image; and a synthesis unit that synthesizes the Chinese character string and the alphabetic string in the image according to the layout to generate a synthesized image.

[0068] According to method 19, the same effect as method 1 can be obtained.

[0069] In the 19th method, the same items as those specified in methods 2 through 15 can be appropriately combined. In this case, the constituent elements responsible for a specific process or function in the image processing method can be grasped as the constituent elements of the image processing system responsible for the corresponding process or function.

[0070] The 20th method can be configured as follows: In the image processing system of the 19th method, there is a printing device that is connected to a network to communicate and perform the printing of composite images.

[0071] According to method 20, it is possible to provide printed materials containing composite images. Thus, it is possible to appreciate Chinese characters using printed materials.

[0072] The 21st method can be configured as follows: In the image processing system of the 20th method, the printing apparatus includes an image analysis unit, a Chinese character string determination unit, an alphabetic string determination unit, a layout determination unit, and a synthesis unit, and performs printing of image-based synthesized images.

[0073] According to the 21st method, the printing apparatus can acquire an image of the object to be processed from the terminal device, generate a composite image based on the acquired image, and provide a printed material with the composite image to the user.

[0074] The 22nd method can be configured as follows: In the image processing system of the 19th or 20th method, it includes: a server device that is connected to a network to enable communication, and includes an image analysis unit, a Chinese character string determination unit, an alphabetic string determination unit, a layout determination unit, and a synthesis unit.

[0075] According to method 22, a server device using a network connection can generate a composite image.

[0076] Invention Effects

[0077] According to the present invention, a string of Chinese characters related to the objects contained in the acquired image and a string of letters related to the meaning of the Chinese characters are combined in the image. Therefore, the content of the Chinese character string can be understood based on the meaning of the letter strings, allowing foreigners and others to easily appreciate Chinese characters. Attached Figure Description

[0078] Figure 1 This is a functional block diagram of the image processing system according to the first embodiment.

[0079] Figure 2 yes Figure 1 The front view of a smartphone involved in a specific example of an image processing device is shown.

[0080] Figure 3 This is a flowchart illustrating the process of the image processing method according to the first embodiment.

[0081] Figure 4 This is an explanatory diagram showing a specific example of a composite image.

[0082] Figure 5 It means extraction Figure 4 An explanatory diagram illustrating a specific example of a composite image when another subject is the object of the image shown.

[0083] Figure 6 It means extraction Figure 4 An explanatory diagram illustrating a specific example of a composite image when another subject is used as the object in the image shown.

[0084] Figure 7 This is an explanatory diagram of a specific example of a Chinese character template database.

[0085] Figure 8 This is a structural diagram of the image processing system according to the second embodiment.

[0086] Figure 9 yes Figure 8 The diagram shown is a functional block diagram of the self-printing terminal device.

[0087] Figure 10 This is a flowchart illustrating the process of the image processing method according to the second embodiment.

[0088] Figure 11 This is a functional block diagram of the layout determination unit in the image processing system according to the third embodiment.

[0089] Figure 12 This is a screen diagram illustrating an example of a layout setting screen in an image processing system according to the third embodiment.

[0090] Figure 13This is a flowchart illustrating the layout determination step in the image processing method according to the third embodiment.

[0091] Figure 14 This is an explanatory diagram of the synthetic image involved in the first specific example.

[0092] Figure 15 This is an explanatory diagram of the synthetic image involved in the second specific example.

[0093] Figure 16 This is an explanatory diagram of the composite image involved in the third specific example.

[0094] Figure 17 This is an explanatory diagram of the composite image involved in the fourth specific example.

[0095] Figure 18 This is an explanatory diagram of the composite image involved in the fifth specific example.

[0096] Figure 19 This is an explanatory diagram of the composite image involved in the sixth specific example.

[0097] Figure 20 This is an explanatory diagram of the composite image involved in the seventh specific example.

[0098] Figure 21 This is an explanatory diagram of the composite image involved in the eighth specific example.

[0099] Figure 22 This is an explanatory diagram of the composite image involved in the ninth specific example.

[0100] Figure 23 This is an explanatory diagram of the composite image involved in the tenth specific example.

[0101] Figure 24 This is an explanatory diagram of the composite image involved in the eleventh specific example.

[0102] Figure 25 This is an explanatory diagram of the composite image involved in the twelfth specific example.

[0103] Figure 26 This is an explanatory diagram of the composite image involved in the thirteenth specific example.

[0104] Symbol Explanation

[0105] 10-Image processing system, 10A-Smartphone, 12-Image acquisition unit, 13-Image storage unit, 14-Object extraction unit, 15-Object attribute information storage unit, 16-Chinese character verification unit, 18-Layout determination unit, 18A-Layout determination unit, 20-Compositing unit, 22-Chinese character template database, 24-Display unit, 24A-Liquid crystal display, 24B-Display unit, 26-Composited image storage unit, 30-Camera device, 42-Object, 42A-Object, 42B-Object, 42C-Object, 42D-Object, 42E-Object, 42F-Object, 42G-Object, 42H-Object, 42I-Object, 42J-Object, 42K-Object, 42L-Object, 42M-Object, 4 2N - Object, 42P - Object, 44 - Chinese character string, 44A - Chinese character string, 44B - Chinese character string, 44C - Chinese character string, 44D - Chinese character string, 44E - Chinese character string, 44F - Chinese character string, 44G - Chinese character string, 44H - Chinese character string, 44I - Chinese character string, 44J - Chinese character string, 44K - Chinese character string, 44L - Chinese character string, 44M - Chinese character string, 44N - Chinese character string, 44P - Chinese character string, 46 - Hash identifier, 46A - Hash identifier, 46B - Hash identifier, 46C - Hash identifier, 46D - Hash identifier, 46E - Hash identifier, 46F - Hash identifier, 46G - Hash identifier, 46H - Hash identifier, 46I - Hash identifier, 46J - Hash identifier 46K - Hash Identifier, 46L - Hash Identifier, 46M - Hash Identifier, 46N - Hash Identifier, 46P - Hash Identifier, 48 - Chinese Character Template, 50 - Image, 50C - Image, 50D - Image, 50E - Image, 50F - Image, 50G - Image, 50H - Image, 50I - Image, 50J - Image, 50K - Image, 50L - Image, 50M - Image, 50N - Image, 50P - Image, 52 - Composite Image, 52A - Composite Image, 52B - Composite Image, 52C - Composite Image, 52D - Composite Image, 52E - Composite Image, 52F - Composite Image, 52G - Composite Image, 52H - Composite Image, 52I - Composite Image, 52 J - Composite image, 52K - Composite image, 52L - Composite image, 52M - Composite image, 52N - Composite image, 52P - Composite image, 56 - First pattern, 58 - Second pattern, 59 - Landscape, 60 - Third pattern, 62 - Figure, 64 - Border, 66 - Figure, 70 - Decoration, 71 - Background of hollowed-out characters, 72 - Fourth pattern, 74 - Fifth pattern, 76 - Sixth pattern, 100 - Image processing system, 102 - Server device, 104 - Network, 106A - First smartphone, 106B - Second smartphone, 108 - Notebook computer, 110 - Self-printing terminal device, 120 - Image acquisition unit, 122 - Image storage unit, 124 - Object extraction unit.126-Object Attribute Information Storage Department, 128-Chinese Character Verification Department, 130-Layout Determination Department, 132-Compositing Department, 134-Chinese Character Template Database, 136-Display Department, 138-Printing Department, 140-Character Size Determination Department, 142-Character Color Determination Department, 144-Character Position Determination Department, 146-Character Direction Determination Department, 148-Character Decoration Determination Department, 150-Foreground Determination Department, 152-Transparency Determination Department, 154-Layout Modification Department, 200-Layout Setting Screen, 202-Compositing Image Display Area, 210-Position Setting Area, 212-X-Coordinate Value Box, 214- y-coordinate value box, 220-size setting area, 222-Chinese character font size box, 224-hash identifier font size box, 230-color setting area, 232-color display box, 234-cutout button, 240-direction setting area, 242-angle box, 250-decoration setting area, 252-button present, 254-no button, 260-foreground setting area, 262-foreground selection button, 264-background selection button, 270-transparency setting area, 272-transparency box, S10 to S28, S100 to S116, S200 to S220-each step of the image processing method. Detailed Implementation

[0106] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification, the same reference numerals are used to denote the same constituent elements, and repeated descriptions are omitted where appropriate.

[0107] [Overall Structure of the Image Processing System]

[0108] Figure 1 This is a functional block diagram of the image processing system according to the first embodiment. The image processing system 10 shown in this diagram combines a Chinese character string related to an object extracted from the image and a letter string related to the meaning of the Chinese character string into the image to generate a composite image. When a signal representing the composite image is sent to a printing terminal device, the printing terminal device generates a printed matter with the composite image printed on it.

[0109] The following example illustrates the use of a single Chinese character as the character string and a hash identifier as the letter string. Furthermore, in this specification, the term "image" refers not only to the image itself but also to image data, which is a signal representing the image. The hash identifier is a label prefixed to the letter string with a hash symbol.

[0110] An alphabet is any set of characters that are used in any language, have a specific order, and belong to an alphabetical system. Examples include the Latin alphabet, the Greek alphabet, and the Cyrillic alphabet.

[0111] The image processing system 10 is compatible with mobile terminal devices such as smartphones. The image processing system 10 is also compatible with computers such as personal computers. The computer can be a desktop computer, a laptop computer, or a tablet computer.

[0112] The image processing system 10 is equipped with application software for generating composite images. The image processing system 10 can execute the application software to perform various functions when generating composite images.

[0113] The image processing system 10 includes an image acquisition unit 12 and an image storage unit 13. The image acquisition unit 12 acquires images captured using a camera device or the like built into a smartphone. The image acquisition unit 12 stores the acquired images in the image storage unit 13.

[0114] The image processing system 10 can also acquire images of the object to be processed from an external storage device. For example, the image acquisition unit 12 can acquire images of the object to be processed via a communication interface (not shown) such as a USB port or a slot for inserting a memory card. USB is an abbreviation for Universal Serial Bus.

[0115] The image processing system 10 includes an object extraction unit 14. The object extraction unit 14 analyzes and processes an image of the object being processed and extracts one or more objects from the image. Object extraction can utilize known image processing techniques. As objects, the object extraction unit 14 can extract the main subject or background components.

[0116] The object extraction unit 14 extracts object attribute information from objects in an image extracted from the processed object. The object attribute information may include the object's type, orientation, size, position in the image, and color. The object extraction unit 14 associates the object attribute information with the image and stores it in the object attribute information storage unit 15. Furthermore, the object extraction unit 14 shown in this embodiment is an example of an image analysis unit.

[0117] The image processing system 10 includes a Chinese character verification unit 16 and a Chinese character template database 22. The Chinese character verification unit 16 refers to the Chinese character template database 22 and uses object attribute information as parameters to verify the Chinese characters representing the content of the object and the hash identifiers representing the meaning of the Chinese characters.

[0118] Furthermore, the Chinese characters representing the content of an object are equivalent to a string of Chinese characters containing one or more characters related to the object. And the hash identifier representing the meaning of the Chinese characters is equivalent to a string of letters related to the meaning of the Chinese character string.

[0119] The Chinese character template database 22 stores combinations of Chinese characters representing subjects captured by cameras such as smartphones, along with hash identifiers representing the meanings of the Chinese characters, and associates them with the subjects. The Chinese character template database 22 can store templates composed of combinations of Chinese characters and hash identifiers, as well as associate them with the subjects.

[0120] That is, the Chinese character verification unit 16 reads a template composed of Chinese characters and hash identifiers corresponding to the object attribute information from the Chinese character template database 22. Additionally, Figure 1 DB, as shown, is an abbreviation for Database.

[0121] The Chinese character template database 22 can be located outside the image processing system 10. The Chinese character template database 22 can be connected to the image processing system 10 via a network or the like to communicate with it.

[0122] The image processing system 10 includes a layout determination unit 18. The layout determination unit 18 uses the Chinese character verification unit 16 to determine the layout of the composite image, such as the position, size, and color of the Chinese character templates read from the Chinese character template database 22 in the image.

[0123] Furthermore, the layout determination unit 18 shown in the embodiment is an example of a kanji string determination unit that determines a kanji string containing one or more kanji characters related to the object. Also, the layout determination unit 18 is an example of an alphabetic string determination unit that determines an alphabetic string related to the meaning of the kanji string.

[0124] The image processing system 10 includes a compositing unit 20. The compositing unit 20 generates a composite image by using a Chinese character template determined by the layout determination unit 18 to composite the image of the object being processed.

[0125] The image processing system 10 includes a display unit 24. The display unit 24 is adapted to the liquid crystal display of a smartphone. The display unit 24 displays a composite image generated using the compositing unit 20.

[0126] The image processing system 10 includes a composite image storage unit 26. When an instruction signal indicating that the composite image should be stored is received, the composite unit 20 stores the composite image in the composite image storage unit 26.

[0127] Hardware structure of an image processing system

[0128] Figure 1The hardware for the various processing units shown can be adapted to various processors. Among these processors, both CPUs (Central Processing Units) and GPUs (Graphics Processing Units) can be used. A CPU is a general-purpose processor that performs various processing functions to execute programs. A GPU is a processor specialized for image processing.

[0129] As various processors, they can be used with devices that include dedicated circuits. Examples of devices that include dedicated circuits include PLDs (Programmable Logic Devices) and ASICs (Application Specific Integrated Circuits). A PLD is a processor whose circuit structure can be changed after the device is manufactured. An ASIC is a processor with a circuit structure specifically designed to perform specific processes.

[0130] A processing unit can be constructed using one of the aforementioned processors, or it can be constructed using two or more processors. As examples of using two or more processors to construct a processing unit, examples of using processors of the same type and examples of using processors of different types can be given.

[0131] As an example of using processors of the same type to form a processing unit, one could cite an example of using two or more FPGAs. As an example of using processors of different types to form a processing unit, one could cite an example of using one or more CPUs and one or more FPGAs to form a processing unit.

[0132] It is also possible to use a single processor to construct multiple processing units. As an example of using a single processor to construct multiple processing units, one can cite a combination of more than one CPU and a program to construct a single processor, and this processor structure functions as multiple processing units. Specific examples of this approach include computers such as server devices and client devices.

[0133] As another example of using a single processor to constitute multiple processing units, one can cite the following method of using a processor that uses an IC chip to implement the overall functionality of a system containing multiple processing units.

[0134] As a specific example of this approach, a system-on-a-chip (SoC) can be cited. Additionally, IC is an abbreviation for Integrated Circuit. SoC is sometimes referred to as a System-on-Chip.

[0135] Various processing units are constructed using one or more processors as their hardware structure. The hardware structure of each processor is a circuit composed of semiconductor components and other circuit elements. A circuit is sometimes referred to as a "circuit" in English.

[0136] [Specific examples of image processing devices]

[0137] Figure 2 yes Figure 1 The figure shows a front view of a smartphone, specifically an example of an image processing device. The smartphone 10A shown uses a liquid crystal display 24A to display the composite image 52. The smartphone 10A has composite image generation application software pre-installed. Hereinafter, the composite image generation application software will be described as an application program.

[0138] The operator of smartphone 10A launches the application. The operator of smartphone 10A activates the camera device 30 built into smartphone 10A to begin taking a picture. The LCD display 24A displays a real-time preview image. Furthermore, the camera device 30 described in this embodiment is an example of an image acquisition unit.

[0139] When the designated subject enters the focal area, a Chinese character string 44 containing one or more Chinese characters corresponding to the subject object 42, and a Chinese character template 48 containing a hash identifier 46 representing the meaning of the Chinese character string 44 are read out. The Chinese character template 48 is composited in the image 50 containing the object 42. The composite image 52 containing the Chinese character template 48 in the image 50 is displayed on the liquid crystal display 24A.

[0140] When a camera button (not shown) is pressed while the composite image 52 is displayed on the LCD 24A, the camera device 30 stores the image 50. The application generates a composite image 52 in which the Chinese character template 48 is composited into the image 50. The application stores the composite image 52.

[0141] [Flowchart of image processing methods]

[0142] Figure 3 This is a flowchart illustrating the process of the image processing method according to the first embodiment. In the application startup step S10, Figure 2 The user of the smartphone 10A activates the application by pressing the operation button on the smartphone 10A. After the application activation step S10, the image processing method proceeds to the shooting start step S12.

[0143] In the shooting start step S12, the operator of the smartphone 10A activates the camera device 30 built into the smartphone 10A by pressing the operation button on the smartphone 10A. The operator of the smartphone 10A points the lens of the camera device 30 toward the subject being photographed. (The illustration of the lens of the camera device 30 is omitted.) The LCD screen 24A of the smartphone 10A displays a real-time preview image of the subject being photographed. After the shooting start step S12, the image processing method proceeds to the subject extraction step S14.

[0144] In the object extraction process S14 Figure 1 The object extraction unit 14 determines whether the specified subject has entered the focal area of ​​the imaging device 30. When the object extraction unit 14 determines in the object extraction process S14 that the specified subject has not entered the focal area of ​​the imaging device 30, it makes a "no" determination. In the case of a "no" determination, the image processing method proceeds to the shooting end determination process S28.

[0145] On the other hand, when the object extraction unit 14 determines that a specified subject has entered the focal area of ​​the imaging device 30 in the object extraction process S14, a "yes" determination is made. In the case of a "yes" determination, the object extraction unit 14 extracts the specified subject as object 42, and the image processing method proceeds to the template reading process S16. Furthermore, the object extraction process S14 shown in the embodiment is an example of an image analysis process.

[0146] In the template reading process S16, the Chinese character verification unit 16 reads the Chinese character template 48 corresponding to the object 42 from the Chinese character template database 22. After the template reading process S16, the image processing method proceeds to the layout determination process S18.

[0147] Furthermore, the template reading process S16 shown in the embodiment is equivalent to an example of a Chinese character string determination process that determines a Chinese character string containing one or more Chinese characters related to the object. Also, the template reading process S16 is equivalent to an example of an alphabetic string determination process that determines an alphabetic string related to the meaning of the Chinese character string.

[0148] In the layout determination step S18, the layout determination unit 18 determines the layout of the composite image 52 based on the Chinese character template 48 and the image 50. After the layout determination step S18, the image processing method proceeds to the composite image generation step S20.

[0149] The layout determination step S18 may include a Chinese character string modification step that modifies the determined Chinese character string 44. Furthermore, the layout determination step S18 may include a hash identifier modification step that modifies the determined hash identifier 46. Additionally, the hash identifier modification step shown in the embodiment is an example of the letter string modification step.

[0150] In the composite image generation step S20, the compositing unit 20 generates a composite image 52 formed by combining the Chinese character template 48 and the image 50. After the composite image generation step S20, the image processing method proceeds to the composite image display step S22.

[0151] In the composite image display step S22, the display unit 24 displays the composite image 52. After the composite image display step S22, the image processing method proceeds to the composite image storage determination step S24.

[0152] In the composite image storage determination process S24, the image processing system 10 determines whether the camera button of the smartphone 10A has been operated since the timing of the composite image being displayed on the display unit 24.

[0153] When the image processing system 10 determines in the composite image storage determination step S24 that the camera button of the smartphone 10A has not been operated since the timing of the composite image being displayed on the display unit 24 and a certain period of time has elapsed, a "No" determination is made. In the case of a "No" determination, the image processing method proceeds to the object extraction step S14. Hereafter, each step from the object extraction step S14 to the composite image storage determination step S24 is repeatedly performed until a "Yes" determination is made in the composite image storage determination step S24.

[0154] On the other hand, when the image processing system 10 determines in the composite image storage determination step S24 that the camera button of the smartphone 10A has been operated within a certain period of time since the composite image was displayed on the display unit 24, it makes a "yes" determination. In the case of a "yes" determination, the image processing method proceeds to the composite image storage step S26.

[0155] In the composite image storage step S26, the compositing unit 20 stores the composite image 52 in the composite image storage unit 26. After the composite image storage step S26, the image processing method proceeds to the shooting end determination step S28.

[0156] In the shooting end determination step S28, the image processing system 10 determines whether to end the shooting. When the image processing system 10 determines in the shooting end determination step S28 that the shooting should not end, it becomes a "no" determination. In the case of a "no" determination, the image processing method proceeds to the object extraction step S14, and repeatedly performs each step from the object extraction step S14 to the shooting end determination step S28, until it becomes a "yes" determination in the shooting end determination step S28.

[0157] On the other hand, when the image processing system 10 determines to end shooting in the shooting end determination step S28, a "Yes" determination is made. In the case of a "Yes" determination, the image processing system 10 ends the image processing method.

[0158] [Specific Example of Composite Image]

[0159] Next, a specific example of the composite image will be described. Figure 4 is an explanatory diagram showing a specific example of the composite image. In addition, Figure 4 The composite image 52 shown is the same as the composite image 52 shown in Figure 2 shown. In Figure 5 there is shown a Chinese character template 48 when the sky is recognized as the subject of the image 50 and the sky is extracted as the object 42. The Chinese character string 44 in the Chinese character template 48 applies to a single-character Chinese character such as "sky". The hash identifier 46 in the Chinese character template 48 applies to the English word "emptiness" representing the meaning of the Chinese character string 44.

[0160] In the composite image 52, the Chinese character string 44 is arranged at a position overlapping with the object 42. The hash identifier 46 is arranged inside the area exclusively occupied by the Chinese character string 44. In the composite image 52, the Chinese character string 44 and the hash identifier 46 are foregrounds, and the image 50 serves as a background. In the composite image 52, a part of the Chinese character string 44 and a part of the hash identifier 46 are subjected to transparency processing, so that the image 50 can be seen through.

[0161] Figure 5 is an explanatory diagram showing a specific example of a composite image when another subject of the image shown in Figure 4 is extracted as an object. In Figure 5 there is shown a composite image 52A when a building in the image 50 is recognized as a subject and extracted as an object 42A. As a result of object analysis, object attribute information indicating that the type of the object is a temple building is extracted for the object 42A.

[0162] Figure 5 The Chinese character string 44A shown applies to a single-character Chinese character "temple" corresponding to the object attribute information. The hash identifier 46A applies to the English word "temple" representing the meaning of the Chinese character string 44A. In the composite image 52A, the Chinese character string 44A is arranged at a position that avoids the temple building as the object 42A and does not overlap with the temple building. The hash identifier 46A is arranged inside the area exclusively occupied by the Chinese character string 44A.

[0163] In the composite image 52A, the Chinese character string 44A and the hash identifier 46A are the foreground, and the image 50 serves as the background. In the composite image 52A, a part of the Chinese character string 44A and a part of the hash identifier 46A are subjected to transparency processing, so that the sky of the image 50 can be seen through.

[0164] Figure 6 is an explanatory diagram showing a specific example of a composite image when another subject in the image shown is extracted as the object 42B. Figure 4 The Figure 6 shows a composite image 52B when the tall tower in the image 50 is recognized as a subject and extracted as the object 42B. As a result of object analysis, object attribute information indicating that the type of the object is a tall tower is extracted for the object 42B.

[0165] Figure 6 The Chinese character string 44B shown uses the single Chinese character "ta" (tower) corresponding to the object attribute information. The hash identifier 46B uses the English word "tower" that represents the meaning of the Chinese character string 44B.

[0166] In the composite image 52B, the Chinese character string 44B is arranged at a position overlapping with the object 42B. Further, the hash identifier 46B is arranged at a position outside the area exclusively occupied by the Chinese character string 44B and avoiding the object 42B. In the composite image 52B, the Chinese character string 44B and the hash identifier 46B are the foreground, and the image 50 serves as the background. In the composite image 52B, a part of the Chinese character string 44B and a part of the hash identifier 46B are subjected to transparency processing, so that the sky, temple building, and tall tower of the image 50 can be seen through.

[0167] [Specific Example of Chinese Character Template Database]

[0168] Figure 7 is an explanatory diagram of a specific example of the Chinese character template database. The Chinese character template database 22 shown in this figure stores the correspondence relationship between object attribute information and the combination of the Chinese character string 44 and the hash identifier 46. Figure 7 The combination of the Chinese character string 44 and the hash identifier 46 shown corresponds to the Figure 2 Chinese character template 48 shown in the drawings and the like.

[0169] In the Chinese character template database 22, one or more Chinese characters are associated with one piece of object attribute information. Further, in the Chinese character template database 22, one or more hash identifiers 46 are associated with one Chinese character string 44. Figure 1 The Chinese character verification unit 16 shown can selectively determine the combination of the Chinese character string 44 and the hash identifier 46 based on one piece of object attribute information.

[0170] The Chinese character template database 22 can verify the combination of Chinese character string 44 and hash identifier 46 by taking object attribute information as a parameter. The Chinese character template database 22 can append object attribute information and append the combination of Chinese character string 44 and hash identifier 46 corresponding to the appended object attribute information.

[0171] Furthermore, the Chinese character template database 22 shown in the embodiment is applicable to the Chinese character string determination process of determining one or more Chinese character strings of a composite object from multiple candidate Chinese character strings representing an object.

[0172] Furthermore, the Chinese character template database 22 shown in the embodiment is applicable to the letter string determination process of determining the letter string of the synthesis object from candidates of multiple letter strings representing the meaning of Chinese character strings.

[0173] [A variation of the Chinese character verification section and template reading process]

[0174] Figure 1 The Chinese character verification section 16 shown is as follows: Figure 3 The template reading process S16 shown can be applied to a learner that has learned the correspondence between object attribute information and Chinese character template 48. That is, the Chinese character verification unit 16 can use a learner that has learned the correspondence between object attribute information and Chinese character string 44 and a learner that has learned the correspondence between Chinese character string 44 and hash identifier 46 to determine the Chinese character template 48 corresponding to the object attribute information.

[0175] The learner is capable of using deep learning methods such as convolutional neural networks. The Chinese character verification unit 16, which uses the learner, can perform relearning using the determined results of the Chinese character template 48.

[0176] [Effects of the image processing system and image processing method according to the first embodiment]

[0177] The image processing system 10 and image processing method according to the first embodiment can achieve the following effects.

[0178] [1]

[0179] When taking a picture using the camera device 30 of the smartphone 10A, the LCD 24A displays a composite image 52, which is a composite image 52 formed by combining the image 50 with the Chinese character string 44 related to the object 42 extracted from the image 50 and a hash identifier related to the meaning of the Chinese character string 44. Thus, foreigners who cannot understand the meaning of Chinese characters can understand the meaning of the Chinese character string 44 based on the hash identifier 46 and can easily appreciate the Chinese characters composited in the image 50.

[0180] [2]

[0181] In the composite image 52, a hash identifier 46 is positioned within the region exclusively occupied by the Chinese character string 44. This allows for the overall visual recognition of the Chinese character string 44 and the hash identifier 46. Furthermore, it enables the perception of a unified representation of the Chinese character string 44 and the hash identifier 46.

[0182] [3]

[0183] In the composite image 52, a Chinese character string 44 and a hash identifier 46 are arranged at a position overlapping with the object 42. This allows for the generation of a composite image 52 with excellent design. Furthermore, it enables the understanding of the correspondence between the object 42 and the Chinese character string 44.

[0184] [4]

[0185] In the composite image 52, the Chinese character string 44 and the hash identifier 46 are set as the foreground, and the object 42 is set as the background. This allows the Chinese character string 44 and the hash identifier 46 to be emphasized in the composite image 52.

[0186] [5]

[0187] Transparency is applied to a portion of the Chinese character string 44 and the hash identifier 46, allowing objects such as object 42 to be visible. Thus, the presence of objects such as object 42 is not compromised in the composite image 52, thereby emphasizing the Chinese character string 44 and the hash identifier 46.

[0188] [6]

[0189] One or more Chinese character strings 44 and hash identifiers 46 can be selected from a combination of multiple Chinese character strings 44 and hash identifiers 46 corresponding to object attribute information. This allows for expanding the selection of Chinese character strings 44 and hash identifiers 46 for an object.

[0190] [Description of the image processing system according to the second embodiment]

[0191] Figure 8 This is a structural diagram of the image processing system according to the second embodiment. The image processing system 100 shown in this figure includes a server device 102, which is connected via a network 104 to communicate with a terminal device.

[0192] exist Figure 8 The image shows multiple smartphones, including a first smartphone 106A and a second smartphone 106B, used as terminal devices. Furthermore, in... Figure 8 The diagram shows a notebook computer 108 and a self-printing terminal device 110 as another terminal device.

[0193] Figure 8 The server device 102 shown is applicable Figure 1The hardware structure of the image processing system 10 is shown. The function of the server device 102 is omitted here. The server device 102 has a communication interface (not shown). The server device 102 is connected via the communication interface to communicate with the network 104.

[0194] Self-printing terminal 110 is installed at tourist locations such as airports, train stations, accommodation facilities, and shops. Self-printing terminal 110 can provide tourists with printing services for images they own.

[0195] [Structural Example of a Self-Printing Terminal Device]

[0196] Figure 9 yes Figure 8 The functional block diagram of the self-printing terminal device is shown. Furthermore, the hardware structure of the self-printing terminal device 110 is adaptable to... Figure 1 The image processing system 10 shown has the same structure.

[0197] Figure 8 The self-printing terminal device 110 shown includes an image acquisition unit 120, an image storage unit 122, an object extraction unit 124, an object attribute information storage unit 126, a Chinese character verification unit 128, a layout determination unit 130, a synthesis unit 132, a Chinese character template database 134, and a display unit 136. Furthermore, the self-printing terminal device 110 includes a printing unit 138.

[0198] The self-printing terminal device 110 includes an image acquisition unit 120, an image storage unit 122, an object extraction unit 124, an object attribute information storage unit 126, a Chinese character verification unit 128, a layout determination unit 130, a synthesis unit 132, a Chinese character template database 134, and a display unit 136. Figure 1 The components of the image processing system 10 shown are identical. Their descriptions are omitted here.

[0199] Figure 9 The printing unit 138 shown performs the printing of composite images. The printing unit 138 can be used with inkjet printing apparatus. The printing unit 138 can also be used with electrophotographic printing apparatus.

[0200] from Figure 8 The first smartphone 106A and other terminal devices shown upload images to the self-printing terminal device 110 via network 104. The self-printing terminal device 110 extracts object and object attribute information from the image, selects a Chinese character template, and determines the layout in the composite image based on the Chinese character template.

[0201] The self-printing terminal device 110 generates a composite image based on the determined layout. The self-printing terminal device 110 prints the composite image.

[0202] Figure 9The Chinese character template database 134 shown can be configured outside the self-printing terminal device 110 and connected via network 104 to communicate with the self-printing terminal device 110.

[0203] It can also be by Figure 8 The server device 102 shown includes an object extraction unit 124, an object attribute information storage unit 126, a Chinese character verification unit 128, a layout determination unit 130, and a synthesis unit 132, which are part of the self-printing terminal device 110.

[0204] Images are uploaded from a terminal device such as a first smartphone 106A to a server device 102 via network 104. The server device 102 generates a composite image based on the uploaded images. The self-printing terminal device 110 can download the composite image, display the composite image using the display unit 136, and print the composite image using the printing unit 138.

[0205] [Flowchart of the image processing method according to the second embodiment]

[0206] Figure 10 This is a flowchart illustrating the process of the image processing method according to the second embodiment. Hereinafter, details related to... Figure 3 The flowchart shown illustrates the repeated steps. In the image acquisition step S100, Figure 9 The image acquisition unit 120 shown acquires images of the object to be processed. For example, the image acquisition unit 120 can acquire images from a memory card, such as a memory card removed from a first smartphone 106A or similar device that has taken images. Furthermore, the image acquisition unit 120 can acquire images from the memory of the camera device via a communication port connected to enable communication with the first smartphone 106A or similar device that has taken images.

[0207] exist Figure 8 In this device, the self-printing terminal device 110 is connected to the network 104 to enable wireless communication with the first smartphone 106A and the like, but the communication between the self-printing terminal device 110 and the first smartphone 106A and the like can also be wired.

[0208] The self-printing terminal device 110 can also acquire images uploaded from the first smartphone 106A or the like to the server device 102 based on instruction signals sent from the first smartphone 106A or the like. The image acquisition unit 120 stores the acquired images in the image storage unit 122. After the image acquisition step S100, the image processing method proceeds to the object extraction step S102.

[0209] In the object extraction step S102, the object extraction unit 124 extracts one or more objects from the acquired image. If the object extraction unit 124 does not extract any objects in the object extraction step S102, a "No" determination is made. In the case of a "No" determination, the image processing method proceeds to the end determination step S116.

[0210] On the other hand, when the object extraction unit 124 extracts more than one object from the image in the object extraction step S102, a "yes" determination is made. In the case of a "yes" determination, the image processing method proceeds to the template reading step S104. After the template reading step S104, the image processing method executes each step in the following order: layout determination step S106, composite image generation step S108, and composite image display step S110.

[0211] Figure 10 The implementation of each step from template reading step S104 to composite image display step S110 is shown below. Figure 3 The template reading process S16 to the composite image display process S22 are the same. After the composite image display process S110, the image processing method proceeds to the printing determination process S112.

[0212] In the printing determination step S112, the printing unit 138 determines whether a printing instruction signal has been acquired. If the printing unit 138 determines in the printing determination step S112 that a printing instruction signal has not been acquired, a "No" determination is made. In the case of a "No" determination, the image processing method proceeds to the end determination step S116.

[0213] As an example of the printing unit 138 determining that no printing instruction signal has been obtained in the printing judgment process S112, the following situation can be cited: a button for selecting whether to perform printing is displayed on the display unit 136, and the button for selecting not to perform printing is operated.

[0214] On the other hand, when the printing unit 138 determines in the printing determination process S112 that a printing instruction signal has been acquired, it is determined to be "yes". In the case of a "yes" determination, the image processing method proceeds to the composite image printing process S114. As an example of the printing unit 138 determining that a printing instruction signal has been acquired in the printing determination process S112, the case where the button for selecting to perform printing displayed on the display unit 136 is operated can be cited.

[0215] In the composite image printing process S114, the printing unit 138 performs the printing of the composite image. The composite image printing process S114 may also include a printing condition setting process, which sets printing conditions such as the size of the printing paper, image quality, and resolution.

[0216] Furthermore, in the printing determination process S112, a process for setting payment conditions for printing can be implemented. Payment conditions can include cash payment, electronic payment, and credit card payment. After the composite image printing process S114, the image processing method proceeds to the completion determination process S116.

[0217] In the termination determination process S116, the self-printing terminal device 110 determines whether the prescribed termination conditions are met. An example of a termination condition is when the termination selection screen for choosing whether to terminate the image processing method is selected.

[0218] If the self-printing terminal device 110 determines in the termination determination step S116 that the prescribed termination conditions are not met, a "No" determination is made. In the case of a "No" determination, the image processing method proceeds to the object extraction step S102. Hereafter, each step from the object extraction step S102 to the termination determination step S116 is repeatedly executed until a "Yes" determination is made in the termination determination step S116.

[0219] On the other hand, when the printing terminal device 110 determines in the end determination process S116 that the prescribed end conditions are met, it becomes a "yes" determination. In the case of a "yes" determination, the printing terminal device 110 performs the prescribed end processing and ends the image processing method.

[0220] When the self-printing terminal device 110 finishes image processing, it can use the display unit 136 to display end messages such as character information indicating that no object was extracted from the image and character information indicating that the image processing has ended.

[0221] [Effects of the image processing system and image processing method according to the second embodiment]

[0222] The image processing system according to the second embodiment can achieve the following effects.

[0223] [1]

[0224] The self-printing terminal device 110 generates a composite image based on an image uploaded from a first smartphone 106A or the like. The self-printing terminal device 110 is able to print the composite image using the printing unit 138. Thus, it is possible to easily appreciate the Chinese characters in the composite image using printed materials.

[0225] [2]

[0226] The self-printing terminal device 110 can download a composite image generated by the server device 102 based on an image uploaded from a first smartphone 106A or the like to the server device 102, and print the downloaded composite image. Thus, it is possible to easily appreciate the Chinese characters in the composite image using printed materials.

[0227] [3]

[0228] The self-printing terminal device 110 can send images uploaded from a first smartphone 106A or the like to a server device 102, download composite images generated using the server device 102, and print the downloaded composite images. Thus, it is possible to easily appreciate the Chinese characters in the composite images using printed materials.

[0229] [Description of the image processing system according to the third embodiment]

[0230] Figure 11 This is a functional block diagram of the layout determination unit in the image processing system according to the third embodiment. The layout determination unit 18A includes a character size determination unit 140, a character font determination unit 141, a character color determination unit 142, a character position determination unit 144, a character direction determination unit 146, a character decoration determination unit 148, a foreground determination unit 150, and a transparency determination unit 152.

[0231] The character size determination unit 140 determines the size of the Chinese character string 44 and the hash identifier 46. The character size determination unit 140 can determine the size of the Chinese character string 44 based on the size of the object contained in the object attribute information. The character size determination unit 140 determines the size of the hash identifier 46 based on the size of the Chinese character string 44.

[0232] The character font determination unit 141 determines the fonts for the Chinese character string 44 and the hash identifier 46. The font can be a standardized font used in the printing industry, or it can be a primitive font. The character font determination unit 141 can individually set the fonts applicable to the Chinese character string 44 and the fonts applicable to the hash identifier 46.

[0233] The character color determination unit 142 determines the colors of the Chinese character string 44 and the hash identifier 46. The character color determination unit 142 can determine the color of the Chinese character string 44 based on the colors of objects contained in the object attribute information. The character color determination unit 142 determines the color of the hash identifier 46 based on the color of the Chinese character string 44.

[0234] The character color determination unit 142 determines whether to apply the hollow character to the Chinese character string 44 and the hash identifier 46. When the hollow character is applied to the Chinese character string 44 and the hash identifier 46, the character color determination unit 142 determines the background color of the hollow character. The hollow character can be made transparent.

[0235] The character position determination unit 144 determines the positions of the Chinese character string 44 and the hash identifier 46. The positions of the Chinese character string 44 and the hash identifier 46 can be determined using coordinates set on the acquired image 50. The character position determination unit 144 can determine the coordinates of the Chinese character string 44 based on the coordinates of the object. The character position determination unit 144 can determine the coordinates of the hash identifier 46 based on the coordinates of the Chinese character string 44. The coordinates of the Chinese character string 44 can be determined using the coordinates of its centroid position. The coordinates of the hash identifier 46 can be determined using the coordinates of its centroid position.

[0236] The character orientation determination unit 146 determines the orientation of the Chinese character string 44 and the hash identifier 46. The character orientation determination unit 146 can use an angle relative to a reference direction set on the image 50 as the orientation of the Chinese character string 44 and the hash identifier 46. The character orientation determination unit 146 can determine the orientation of the Chinese character string 44 based on the orientation of the object. The character orientation determination unit 146 can determine the orientation of the hash identifier 46 based on the orientation of the Chinese character string 44.

[0237] The character decoration determination unit 148 determines whether to perform decoration processing on the Chinese character string 44 and the hash identifier 46. The character decoration determination unit 148 can perform decoration processing on the Chinese character string 44 based on its content. The character decoration determination unit 148 can perform decoration processing on the hash identifier 46 based on the decoration processing performed on the Chinese character string 44.

[0238] The foreground determination unit 150 determines whether to set the Chinese character string 44 and the hash identifier 46 as the foreground or the background. The foreground determination unit 150 can also change the initial setting of setting the Chinese character string 44 and the hash identifier 46 as the foreground to setting the Chinese character string 44 and the hash identifier 46 as the background.

[0239] When the Chinese character string 44 and hash identifier 46 are in the foreground, the transparency determination unit 152 sets the transparency level of the Chinese character string 44 and hash identifier 46. The transparency determination unit 152 can use transparency as an index value to represent the transparency level. When the Chinese character string 44 and hash identifier 46 are cutout characters, the transparency determination unit 152 determines the transparency of the background of the cutout characters.

[0240] Regarding transparency, the system can set the state where the foreground does not allow the background to see through to 0%, and the state where the foreground is transparent to 100%. The transparency determination unit 152 can set the transparency of the Chinese character string 44 and the hash identifier 46 to a value greater than 0% and less than 100%. The transparency determination unit 152 can also set the transparency of the background of the cutout characters to a value greater than 0% and less than 100%.

[0241] The layout determination unit 18A includes a layout modification unit 154. When the image processing system 10 receives an instruction signal indicating layout modification, the layout modification unit 154 performs the layout modification.

[0242] The layout modification unit 154 can modify character size, character color, character position, character direction, and character decoration. The layout modification unit 154 can also modify the foreground and background, as well as the transparency of the foreground.

[0243] Figure 12 This is a diagram illustrating an example of a layout setting screen in the image processing system according to the third embodiment. The layout setting screen 200 shown in this diagram is displayed on the display unit 24B when the layout is manually set. The display unit 24B is compatible with a liquid crystal display device connected to a computer.

[0244] The operator can visually recognize the layout setting screen 200 and operate the keyboard and mouse (not shown) and other operating components to set the layout of the composite image. Furthermore, the term "setting" as used here can include the concept of modification.

[0245] The layout setting screen 200 includes a composite image display area 202. The composite image display area 202 displays the image 50 of the object being processed, the Chinese character string 44, and the hash identifier 46.

[0246] The layout setting screen 200 includes a position setting area 210. The position setting area 210 includes an x-coordinate value box 212 that displays the x-coordinate value and a y-coordinate value box 214 that displays the y-coordinate value.

[0247] The layout setting screen 200 includes a size setting area 220. The size setting area 220 includes a Chinese character size box 222 that displays the size of the Chinese character string 44 and a hash identifier size box 224 that displays the size of the hash identifier 46.

[0248] The layout settings screen 200 includes a color settings area 230. The color settings area 230 includes a color display box 232 that displays the color and a cutout button 234 that indicates whether cutout characters are applicable. The color display box 232 can display either the color itself or a string representing the color. When the cutout button 234 is selected, the color display box 232 displays the background color of the cutout characters.

[0249] The layout setting screen 200 includes an orientation setting area 240. The orientation setting area 240 includes an angle box 242 that displays the value of the angle relative to a reference direction. Regarding the angle relative to the reference direction, the reference angle can be set to 0 degrees, the clockwise direction can be set to the positive direction, and the counterclockwise direction can be set to the negative direction.

[0250] The layout setting screen 200 includes a decoration setting area 250 for setting the presence or absence of decorations. The decoration setting area 250 includes a "Yes" button 252 for when implementing decorations and a "No" button 254 for when not implementing decorations.

[0251] The layout setting screen 200 includes a foreground setting area 260 for setting whether the Chinese character string 44 and the hash identifier 46 are set as the foreground or the background. The foreground setting area 260 includes a foreground selection button 262 for setting the Chinese character string 44 and the hash identifier 46 as the foreground and a background selection button 264 for setting the Chinese character string 44 and the hash identifier 46 as the background. Figure 12 The layout setting screen 200 shown indicates the state where the Chinese character string 44 and the hash identifier 46 are set as the foreground.

[0252] The layout settings screen 200 includes a transparency setting area 270 for setting the transparency of the Chinese character string 44 and the hash identifier 46 when they are set as the foreground characters. The transparency setting area 270 includes a transparency box 272 that displays the transparency value.

[0253] The x-coordinate value box 212 can be used for inputting numerical values ​​or for displaying drop-down menus. The layout settings screen 200 can also display the initial settings for the composition parameters.

[0254] Figure 12 The layout setting screen 200 shown can correspond to Figure 2 The display, as shown in the smartphone 10A, has been reduced in size to accommodate different functions. Furthermore, it can be configured to create pages for each setting item, allowing users to scroll through multiple pages while making each setting.

[0255] Figure 13 This is a flowchart illustrating the layout determination step in the image processing method according to the third embodiment. Figure 13 The processes shown are applicable to Figure 3 The layout determination process S18 is shown. In the temporary synthesis process S200, Figure 11 The layout determination unit 18A shown combines the image 50 and the Chinese character template 48 to generate a temporary composite image. After the temporary composite process S200, the layout determination process proceeds to the temporary display process S202.

[0256] In the temporary display process S202, the layout determination unit 18A uses... Figure 2 The liquid crystal display 24A shown displays a temporary composite image. After the temporary display process S202, the layout determination process proceeds to the setting determination process S204. In the setting determination process S204, the layout determination unit 18A determines whether to perform layout setting on the temporary composite image.

[0257] When the layout determination unit 18A determines in the setting determination process S204 that no setting will be performed on the temporary composite image, it becomes a "No" determination. In the case of a "No" determination, the layout determination unit 18A ends the layout determination process and proceeds to... Figure 3 The image synthesis generation process S20.

[0258] On the other hand, when the layout determination unit 18A determines in the setting determination process S204 that a setting is to be performed on the temporary composite image, it becomes a "yes" determination. In the case of a "yes" determination, the layout determination process proceeds to the size determination process S206.

[0259] In the setting determination process S204, the layout determination unit 18A can use the liquid crystal display 24A to display a selection screen that allows users to select whether to set the temporary composite image. The user can operate the selection screen to instruct the setting of the temporary composite image.

[0260] In the size determination step S206, the character size determination unit 140 determines the size of the Chinese character template 48. After the size determination step S206, the image processing method proceeds to the color determination step S208. In the color determination step S208, the character color determination unit 142 determines the color of the Chinese character template 48.

[0261] In the color determination process S208, the character color determination unit 142 determines whether to apply a cutout character to the Chinese character template 48. When a cutout character is applied to the Chinese character template 48, the character color determination unit 142 determines the background color of the cutout character.

[0262] As an example of applying hollow characters to the Chinese character template 48 in the color determination process S208, a case can be given in which the user specifies the application of hollow characters to the Chinese character template 48 using a user interface.

[0263] That is, the color determination process S208 can include a determination process for whether the hollow character is applicable to the Chinese character template 48 and a background color determination process for determining the background color of the hollow character. After the color determination process S208, the layout determination process proceeds to the position determination process S210.

[0264] In the position determination step S210, the character position determination unit 144 determines the position of the Chinese character template 48. After the position determination step S210, the image processing method proceeds to the orientation determination step S212. In the orientation determination step S212, the character orientation determination unit 146 determines the orientation of the Chinese character template 48. After the orientation determination step S212, the layout determination step proceeds to the decoration determination step S214.

[0265] In the decoration determination step S214, the character decoration determination unit 148 determines whether the decoration of the Chinese character template 48 is present or absent. When the decoration of the Chinese character template 48 is performed, the character decoration determination unit 148 determines the content of the decoration. After the decoration determination step S214, the layout determination process proceeds to the foreground and background determination step S216.

[0266] In the foreground / background determination process S216, the foreground determination unit 150 determines whether to set the Chinese character template 48 as the foreground or the background. After the foreground / background determination process S216, the layout determination process proceeds to the transparency determination process S218.

[0267] In the transparency determination step S218, the transparency determination unit 152 determines the transparency of the Chinese character template 48. After the transparency determination step S218, the image processing method proceeds to the layout confirmation determination step S220. When the layout determination unit 18A determines in the layout confirmation determination step S220 that layout modification is required, a "No" determination is made. In the case of a "No" determination, the layout determination process proceeds to the size determination step S206. Thereafter, each step from the size determination step S206 to the layout confirmation determination step S220 is repeated until a "Yes" determination is made in the layout confirmation determination step S220.

[0268] On the other hand, when the layout determination unit 18A determines in the layout confirmation and determination process S220 that no layout modification is required, it becomes a "yes" determination. In the case of a "yes" determination, the layout determination unit 18A ends the layout setting process and proceeds to... Figure 3 The image synthesis generation process S20.

[0269] exist Figure 13 In the flowchart shown, the steps from the size determination step S206 to the layout confirmation and judgment step S220 can be omitted as appropriate. Figure 13 The flowchart for setting up the layout shown can also be applied to the flowchart when modifying the layout.

[0270] In this embodiment, the unit for determining size and position is set as Chinese character template 48, but the size and position can also be determined individually for each Chinese character string and letter string.

[0271] [Effects of the image processing apparatus and image processing method according to the third embodiment]

[0272] The image processing apparatus and image processing method according to the third embodiment can achieve the following effects.

[0273] [1]

[0274] It is possible to determine the size, color, position, and the like of characters. Accordingly, a composite image with excellent design properties can be generated. Furthermore, it is possible to realize emphasis of a Chinese character string, color coordination in the composite image, and the like.

[0275] 〔2〕

[0276] The layout of the composite image can be modified. Accordingly, a composite image that reflects the user's preferences, tastes, and the like can be generated.

[0277] [Specific Examples of Composite Image]

[0278] 〔First Specific Example〕

[0279] Using Figures 14 to 22 specific examples of the composite image will be described. Figure 14 is an explanatory diagram of the composite image according to the first specific example. The composite image 52C shown in this diagram uses an image 50C in which a Buddha statue arranged in a Buddhist altar is photographed.

[0280] In the composite image 52C, the Chinese character string "Zen" related to the Buddha statue extracted as the object 42C is composited onto the image 50C as the Chinese character string 44C. Further, in the composite image 52C, #zen related to the Chinese character string 44C such as "Zen" is composited onto the image 50C as the hash identifier 46C.

[0281] In the composite image 52C, the first pattern 56 that evokes the association of the object 42C is composited onto the Chinese character string 44C. Further, in the composite image 52C, the second pattern 58 that evokes the association of the object 42C is composited onto the object 42C.

[0282] The first pattern 56 and the second pattern 58 use the same color as the Chinese character string 44C. In addition, in this specification, the same color may include modes with different lightness and density.

[0283] The Chinese character string 44C uses decorative characters with stepped oblique sides. On the other hand, the hash identifier 46C uses ordinary characters. That is, the Chinese character string 44C and the hash identifier 46C use different styles.

[0284] In the composite image 52C according to the first specific example, the first pattern 56 and the second pattern 58 that evoke the association of the object 42C are imparted to the image 50C. Accordingly, the image of the object 42C can be appreciated based on the Chinese character string 44C, the first pattern 56, and the second pattern 58.

[0285] Further, the Chinese character string 44C, the hash identifier 46C, the first pattern 56, and the second pattern 58 are set to the same color. Accordingly, a sense of unity of the Chinese character string 44C, the hash identifier 46C, the first pattern 56, and the second pattern 58 can be expressed.

[0286] 〔Second Specific Example〕

[0287] Figure 15 is an explanatory diagram of a composite image according to the second specific example. The composite image 52D shown in this figure uses an image 50D in which multiple smiling people are photographed against a city landscape 59 as a background. The image 50D can be generated by setting the city landscape 59 as the background, setting the multiple people as the foreground, and compositing the two. In the composite image 52D, the Chinese character string 44D of "laugh" associated with the multiple smiling people extracted as the object 42D is composited onto the image 50D. Further, in the composite image 52D, the hash identifier 46D of #SMILE associated with the Chinese character string 44D of "laugh" is composited onto the image 50D.

[0288] That is, in the composite image 52D, the Chinese character string 44D is cut along the contour of the city landscape 59 that is the background of the object 42D. That is, in the composite image 52D, the object 42D and the city landscape 59 are set as the foreground, and the Chinese character string 44D and the hash identifier 46D are set as the background.

[0289] Decorative characters with white borders are applied to the Chinese character string 44D. The same color as the Chinese character string 44D is applied to the hash identifier 46D. Further, the same color as the border of the Chinese character string 44D is applied to the background color of the hash identifier 46D.

[0290] In the composite image 52D according to the second specific example, the Chinese character string 44D associated with the object 42D can be viewed without impairing the atmosphere of the image 50D.

[0291] [Third Specific Example]

[0292] Figure 16 is an explanatory diagram of a composite image according to the third specific example. The composite image 52E shown in this figure uses an image 50E in which ramen is photographed. In the composite image 52E, the Chinese character string 44E of "noodle" associated with the ramen extracted as the object 42E is composited onto the image 50E.

[0293] Further, in the composite image 52E, the hash identifier 46E of #ramen associated with the Chinese character string 44E of "noodle" is composited onto the image 50E. In addition, in the composite image 52E, the third pattern 60 imitating the Chinese characters of "photograph" is composited on the lower edge of the composite image 52E with respect to the image 50E. The third pattern 60 is a pattern imitating the style of the pattern represented on the outer peripheral surface of the ramen bowl of the object 42E.

[0294] The Chinese character string 44E is applicable to hollow-out characters. The hash identifier 46E and the third pattern 60 are also applicable to hollow-out characters. The background of the Chinese character string 44E and the like can be a background color of any color, or a mesh pattern of any color. In addition, in this specification, the term "transparent" can encompass the concept of translucent.

[0295] In the composite image 52E, the background of the image 50E is visible through the Chinese character string 44E that uses relatively thick hollow-out characters. Moreover, in the composite image 52E, the object 42E can be reliably viewed through the Chinese character string 44E.

[0296] In addition, in the composite image 52E, at least a part of the contour of the object 42E is cut out and arranged outside the contour of the Chinese character string 44E. Furthermore, the background of the Chinese character string 44E is translucent, so that the background of the image 50E can be faintly seen.

[0297] With the composite image 52E according to the third specific example, the Chinese character string 44E related to the object 42E can be appreciated without impairing the atmosphere of the image 50E.

[0298] [Fourth Specific Example]

[0299] Figure 17 This is an explanatory diagram of a composite image according to the fourth specific example. The composite image 52F shown in this diagram applies an image 50F capturing the station building of Tokyo Station. In the composite image 52F, a two-character word "東京" related to the station building of Tokyo Station extracted as the object 42F is composited on the image 50F as the Chinese character string 44F. Furthermore, in the composite image 52F, #tokyo related to the Chinese character string 44F of "東京" is composited on the image 50F as the hash identifier 46F.

[0300] In the composite image 52F, the Chinese character string 44F is arranged at a position that avoids the object 42F and contacts the outer side of the object 42F. The position contacting the outer side of the object 42F may include a position overlapping with a part of the object 42F.

[0301] In the composite image 52F, a plurality of Chinese character strings 44F are arranged, and the plurality of Chinese character strings 44F are arranged at positions sandwiching the object 42F from the left and right sides. The Chinese character strings 44F adopt colors extracted from the object 42F.

[0302] With the composite image 52F according to the fourth specific example, the Chinese character string 44F can be emphasized, and overall color harmony can be achieved.

[0303] [Fifth Specific Example]

[0304] Figure 18is an explanatory diagram of a composite image according to the fifth specific example. The composite image 52G shown in this diagram uses an image 50G in which a goldfish swimming in a water tank is photographed. In the composite image 52G, the Chinese character string 44G of "fish" related to the goldfish extracted as the object 42G is composited onto the image 50G. Further, in the composite image 52G, the hash identifier 46G of #fish related to the Chinese character string 44G such as "fish" is composited onto the image 50G.

[0305] In the composite image 52F, the Chinese character string 44G is composited for each of the plurality of objects 42G of the same type. In the composite image 52F, the hash identifier 46G is not composited for some of the objects 42G.

[0306] In the composite image 52F, the Chinese character string 44G is rotated corresponding to the orientation of the object 42G. That is, in the composite image 52F, the orientation of the object 42G is aligned with the orientation of the Chinese character string 44G. Further, in the composite image 52F, the Chinese character string 44G is arranged on the contour line of the object 42G while avoiding the center position of the object 42G.

[0307] In the composite image 52G according to the fifth specific example, the Chinese character string 44G can be emphasized, and a sense of unity between the object 42G and the Chinese character string 44G can be expressed.

[0308] [Sixth Specific Example]

[0309] Figure 19 is an explanatory diagram of a composite image according to the sixth specific example. The composite image 52H shown in this diagram uses an image 50H in which a person 62 is photographed with a plurality of cat ornaments as the background. In the composite image 52H, the Chinese character string 44H of "cat" related to the plurality of cat ornaments extracted as the objects 42H is composited onto the image 50H. Further, in the composite image 52H, the hash identifier 46H of #cat related to the Chinese character string 44H such as "cat" is composited onto the image 50H.

[0310] In the composite image 52H, a frame 64 in which Chinese characters identical to the Chinese character string 44H are continuously arranged along the edge is composited.

[0311] In the composite image 52H according to the sixth specific example, the Chinese character string 44H can be emphasized, and the design performance is excellent.

[0312] [Seventh Specific Example]

[0313] Figure 20is an explanatory diagram of a composite image according to the seventh specific example. The composite image 52I shown in this diagram applies an image 50I in which a plurality of persons 66 are photographed with Mount Fuji as the background. In the composite image 52I, a word "富士山" composed of three Chinese characters related to Mount Fuji extracted as the object 42I is composited on the image 50I as a Chinese character string 44I. Further, in the composite image 52I, #fuji related to the Chinese character string 44I such as "富士山" is composited on the image 50I as a hash identifier 46I.

[0314] In the composite image 52I, the Chinese character string 44I is arranged at the left-right center position relative to the image 50I. That is, in the composite image 52I, the Chinese character string 44I is arranged at the center position of the image 50I along the arrangement direction of the characters constituting the Chinese character string 44I.

[0315] In the composite image 52I according to the seventh specific example, the presence of the object 42I in the image 50I can be maintained, and the overall design performance is excellent.

[0316] [Eighth Specific Example]

[0317] Figure 21 is an explanatory diagram of a composite image according to the eighth specific example. The composite image 52J shown in this diagram applies an image 50J in which a plurality of persons posing cheerfully are photographed. The image 50J may be synthesized by taking the plurality of persons as the foreground and a tall tower as the background.

[0318] In the composite image 52J, a word "楽" related to the plurality of cheerfully posing persons extracted as the object 42J is composited on the image 50J as a Chinese character string 44J. Further, in the composite image 52J, #fun related to the Chinese character string 44J such as "楽" is composited on the image 50J as a hash identifier 46J.

[0319] In the composite image 52J, the Chinese character string 44J is set to be translucent, so that the background of the image 50J can be seen through. In the composite image 52J, the object 42J is defined as the foreground, and the Chinese character string 44J is defined as the background.

[0320] In the composite image 52J according to the eighth specific example, the presence of the object 42J in the image 50J can be maintained, and the background of the image 50J can be visually recognized.

[0321] [Ninth Specific Example]

[0322] Figure 22is an explanatory diagram of the composite image according to the ninth specific example. The composite image 52K shown in this figure applies the image 50K capturing a memorial archway. In the composite image 52K, the word "神" related to the memorial archway extracted as the object 42K is composited on the image 50K as the Chinese character string 44K. Further, in the composite image 52K, #god related to the Chinese character string 44K of "神" is composited on the image 50K as the hash identifier 46K.

[0323] In the composite image 52K, corresponding to the bilaterally symmetric object 42K, the center position of the image 50K in the left-right direction is aligned and arranged with the center position of the Chinese character string 44K. Further, the Chinese character string 44K is provided with a decoration 70 representing the atmosphere of the Chinese character string 44K itself. In addition, in the composite image 52K, the center position of the image 50K in the left-right direction is aligned and arranged with the center position of the hash identifier 46K.

[0324] In the composite image 52K according to the ninth specific example, a design that emphasizes the sense of unity among the object 42K, the Chinese character string 44K, and the hash identifier 46K and places importance on object nature is adopted.

[0325] [Tenth Specific Example]

[0326] Figure 23 is an explanatory diagram of the composite image according to the tenth specific example. The composite image 52L shown in this figure applies the image 50L capturing sushi served on a plate. In the composite image 52L, the word "鮨" related to the sushi extracted as the object 42L is composited on the image 50L as the Chinese character string 44L. Further, in the composite image 52L, #SUSHI related to the Chinese character string 44L of "鮨" is composited on the image 50L as the hash identifier 46L.

[0327] Hollow characters are applied to the Chinese character string 44L. The color used in the object 42L is applied to the background 71 of the hollow characters. Further, in the composite image 52L, two hash identifiers 46L are applied to one Chinese character string 44L. In the composite image 52L, the center of the Chinese character string 44L is arranged at the center position of the composite image 52L, and the two hash identifiers 46L are arranged on the left and right sides relative to the Chinese character string 44L. The two hash identifiers 46L are set to orientations opposite to each other.

[0328] The composite image 52L according to the tenth specific example adopts a design that has a sense of color unity as a whole and places importance on object nature. Uppercase letters can be applied to the letters used for the hash identifier 46L. It is also possible to make only the first letter uppercase for the letters applied to the hash identifier 46L. A plurality of hash identifiers 46L can be composited for one Chinese character string 44L.

[0329] [Eleventh Specific Example]

[0330] Figure 24 is an explanatory diagram of a composite image related to the eleventh specific example. The composite image 52M shown in this drawing applies an image 50M formed by compositing an image capturing a person sitting on a postbox and an image capturing a scenery of a busy street in the daytime. In the composite image 52M, a word "ren" (meaning endurance) related to the person extracted as the object 42M is composited onto the image 50M as a Chinese character string 44M. Further, in the composite image 52M, #ninja related to the meaning of the Chinese character string 44M "ren" is composited onto the image 50M as a hash identifier 46M.

[0331] In the composite image 52M, with respect to the image 50M, at the lower edge of the composite image 52M, composited is a pattern identical to Figure 16 the third pattern 60 shown in the drawing, which is the fourth pattern 72.

[0332] In the composite image 52M, the person sitting on the postbox that constitutes the image 50M is at the forefront, and the scenery of the busy street in the daytime that constitutes the image 50M is at the rearmost position in the composite image 52M.

[0333] The composite image 52M related to the eleventh specific example applies an image formed by compositing two images as the image 50M, where one of the two images is at the forefront and the other is at the rearmost position. In this way, various images can be composited to appreciate Chinese characters.

[0334] [Twelfth Specific Example]

[0335] Figure 25 is an explanatory diagram of a composite image related to the twelfth specific example. The composite image 52N shown in this drawing applies an image 50N capturing a person walking on a shrine approach path. In the composite image 52N, a word "bu" (meaning walk) related to the person walking on the shrine approach path extracted as the object 42N is composited onto the image 50N as a Chinese character string 44N. Further, in the composite image 52N, #walking related to the meaning of the Chinese character string 44N "bu" is composited onto the image 50N as a hash identifier 46N.

[0336] In the composite image 52N, the Chinese character string 44N is arranged without spatial incongruity by spatial deformation. The Chinese character string 44N is deformed according to the shape of the shrine approach path in the image 50N.

[0337] The composite image 52N related to the twelfth specific example applies deformation and arrangement of the Chinese character string 44N that has no spatial incongruity, and has excellent design performance.

[0338] [Thirteenth Specific Example]

[0339] Figure 26is an explanatory diagram of a composite image according to the thirteenth specific example. The composite image 52P shown in this diagram applies the image 50P capturing a couple. In the composite image 52P, a word "Ai" (meaning love) related to the couple extracted as the object 42P is composited onto the image 50P as a Chinese character string 44P. Furthermore, in the composite image 52P, #love related to the meaning of the Chinese character string 44P "Ai" is composited onto the image 50P as a hash identifier 46P.

[0340] In the composite image 52P, the Chinese character strings 44P are respectively arranged at the positions of the shoulders of the two persons, avoiding the faces of the object 42P. The fifth pattern 74 is composited in the background of one of the two Chinese character strings 44P, and the sixth pattern 76 is composited in the background of the other. The hash identifier 46P is arranged only at the position overlapping the fifth pattern 74 among the two Chinese character strings 44P.

[0341] In the composite image 52P according to the thirteenth specific example, a plurality of Chinese character strings 44P are arranged while avoiding the faces of the object 42P. Thereby, a sense of unity between the object 42P and the Chinese character strings 44P can be expressed. Furthermore, the fifth pattern 74 and the sixth pattern 76 are overlapped on the Chinese character strings 44P. Thereby, the Chinese character strings 44P with excellent design properties can be applied.

[0342] [Application example of a program that causes a computer to function as an image processing system]

[0343] The above-described image processing method and image processing system can be configured as a program that uses a computer to implement functions corresponding to the respective steps in the image processing method or the respective steps in the image processing system.

[0344] For example, a program can be configured to implement the following functions: an image analysis function of analyzing the acquired image 50 and extracting the object 42; a Chinese character string determination function of determining a Chinese character string related to the object; an alphabetic string determination function of determining an alphabetic string related to the meaning of the Chinese character; a layout determination function of determining the layout of the composite image; and a compositing function of generating the composite image according to the layout.

[0345] The program that causes a computer to implement the above-described image processing functions can be stored in a computer-readable information storage medium, which is a tangible non-transitory information storage medium, and the program is provided via the information storage medium.

[0346] Furthermore, instead of the mode in which the program is stored in a non-transitory information storage medium for provision, it is also possible to adopt a mode in which a program signal is provided via a communication network.

[0347] [Combination of embodiments, modified examples, and the like]

[0348] The constituent elements described in the above embodiments and the modified embodiments can be appropriately combined and used, and some constituent elements can also be replaced.

[0349] The embodiments of the present invention described above can be appropriately modified, added to, or deleted without departing from the spirit of the present invention. The present invention is not limited to the embodiments described above, and many modifications can be made by those skilled in the art within the scope of the technical concept of the present invention.

Claims

1. An image processing method, characterized in that, include: The image analysis process analyzes the acquired image and extracts non-character objects from the image; The process of determining Chinese character strings involves determining Chinese character strings that contain one or more Chinese characters related to the non-character object. The letter string determination process involves determining a letter string that is related to the meaning of the Chinese character string. The template reading process reads out the Chinese character template containing the Chinese character string and the letter string corresponding to the object; The layout determination process involves determining, based on the Chinese character template and the image, the layout in which the Chinese character string and the letter string related to the object are combined in the image containing the object. and The compositing process involves synthesizing the Chinese character string and the letter string related to the object into the image containing the object based on the layout, thereby generating a composite image. The image processing method further includes: a decoration determination step, which involves performing decoration processing on the Chinese character string based on its content, and performing decoration processing on the letter string based on the decoration processing of the Chinese character string.

2. The image processing method according to claim 1, wherein, The decoration process applied to letter strings is the same as that applied to Chinese character strings.

3. The image processing method according to claim 1 or 2, wherein, The layout determination process involves configuring the layout of the letter string in the area where the Chinese character string is configured.

4. The image processing method according to claim 1 or 2, wherein, The layout determination process employs a layout in which at least a portion of the Chinese character string is positioned to overlap with the object.

5. The image processing method according to claim 1 or 2, wherein, The layout determination process involves placing the Chinese character string in the foreground of the object.

6. The image processing method according to claim 1 or 2, wherein, The layout determination process employs a layout that places the object in the foreground of the Chinese character string.

7. The image processing method according to claim 1 or 2, wherein, The layout determination process employs a layout that allows the object to be visible for at least a portion of the Chinese character string.

8. The image processing method according to claim 1 or 2, wherein, The image analysis process extracts the colors used in the object. The layout determination process uses the extracted color as the color of the Chinese character string.

9. The image processing method according to claim 1 or 2, wherein, The image analysis process extracts multiple objects from the image. The layout determination process involves configuring the Chinese character string in each of the plurality of objects.

10. The image processing method according to claim 1 or 2, wherein, The layout determination process involves configuring the Chinese character string to an orientation that corresponds to the orientation of the object.

11. The image processing method according to claim 1 or 2, wherein, The layout determination process involves placing one or more Chinese characters contained in the Chinese character string at the edge of the image.

12. The image processing method according to claim 1 or 2, wherein, The Chinese character string determination process determines one or more Chinese character strings of a composite object from multiple candidate Chinese character strings related to the object.

13. The image processing method according to claim 1 or 2, wherein, The letter string determination process determines the letter string of the synthesized object from multiple candidate letter strings that are related to the meaning of the Chinese character string.

14. The image processing method according to claim 1 or 2, wherein, include: The Chinese character string modification process modifies the Chinese character string determined in the Chinese character string determination process into another Chinese character string related to the object.

15. The image processing method according to claim 1 or 2, wherein, include: The letter string modification process modifies the letter string determined in the letter string determination process into another letter string that is related to the meaning of the Chinese character string.

16. The image processing method according to claim 1 or 2, wherein, include: The printing process involves using a printing device to print the composite image.

17. A recording medium having a program recorded thereon, characterized in that, The program enables the computer to perform the following functions: Image analysis function, which analyzes the acquired image and extracts non-character objects from the image; The function for determining Chinese character strings is to identify Chinese character strings that contain one or more Chinese characters related to the non-character object. The letter string determination function identifies letter strings that are related to the meaning of the Chinese character string. The template reading function reads out Chinese character templates containing the Chinese character string and the letter string corresponding to the object. The layout determination function, based on the Chinese character template and the image, determines the layout when combining the Chinese character string and the letter string related to the object into the image containing the object; and The compositing function, based on the layout, synthesizes the Chinese character string and the letter string related to the object in the image containing the object to generate a composite image. The program enables the computer to: The decoration determination function applies decoration processing to the Chinese character string based on its content, and applies decoration processing to the letter string based on the decoration processing of the Chinese character string.

18. The recording medium according to claim 17, wherein, The decoration process applied to letter strings is the same as that applied to Chinese character strings.

19. An image processing system, characterized in that, have: The image analysis unit analyzes the acquired image and extracts non-character objects from the image; The Chinese character string determination section determines a Chinese character string that contains one or more Chinese characters related to the non-character object. The letter string determination unit determines a letter string that is related to the meaning of the Chinese character string. The template reading unit reads out Chinese character templates containing the Chinese character string and the letter string corresponding to the object; The layout determination unit determines, based on the Chinese character template and the image, the layout when combining the Chinese character string and the letter string related to the object into the image containing the object; and The compositing unit generates a composite image by compositing the Chinese character string and the letter string related to the object in the image containing the object according to the layout. The image processing system further includes: a decoration determination unit, which performs decoration processing on the Chinese character string based on its content, and performs decoration processing on the letter string based on the decoration processing of the Chinese character string.

20. The image processing system according to claim 19, wherein, The decoration process applied to letter strings is the same as that applied to Chinese character strings.

21. The image processing system according to claim 19, wherein, It includes a printing unit for printing the composite image.

22. An image processing system, characterized in that, They can communicate via a network connection. The image processing system includes: The image analysis unit analyzes the acquired image and extracts non-character objects from the image; The Chinese character string determination section determines a Chinese character string that contains one or more Chinese characters related to the non-character object. The letter string determination unit determines a letter string that is related to the meaning of the Chinese character string. The template reading unit reads out Chinese character templates containing the Chinese character string and the letter string corresponding to the object; The layout determination unit determines, based on the Chinese character template and the image, the layout when combining the Chinese character string and the letter string related to the object into the image containing the object; and The compositing unit generates a composite image by compositing the Chinese character string and the letter string related to the object in the image containing the object according to the layout. The image processing system further includes: a decoration determination unit, which performs decoration processing on the Chinese character string based on its content, and performs decoration processing on the letter string based on the decoration processing of the Chinese character string.

23. The image processing system according to claim 22, wherein, The decoration process applied to letter strings is the same as that applied to Chinese character strings.

24. The image processing system according to claim 22, wherein, It includes: a printing apparatus connected via the network to enable communication and to print the composite image.

25. The image processing system according to claim 24, wherein, The printing apparatus includes the image analysis unit, the Chinese character string determination unit, the letter string determination unit, the layout determination unit, and the synthesis unit, and performs printing of the synthesized image based on the image.

26. The image processing system according to any one of claims 22 to 24, wherein, It includes: a server device, which is connected to the network to communicate, and includes the image analysis unit, the Chinese character string determination unit, the letter string determination unit, the layout determination unit, and the synthesis unit.

Images