Image processing method, image processing system, and image processing program

Abstract

This invention provides an image processing method for captured images that can suppress misidentification of markers. [Solution] An image processing method according to one aspect of the present disclosure generates a lowlight layer from a captured image of a marker that emphasizes a portion of a dissimilar color that is not similar to the color of the marker, generates a highlight layer from the captured image that emphasizes a portion of a similar color that is the same as or similar to the color of the marker, generates an inverted lowlight layer by inverting the lowlight layer, and makes the portion of the marker in the captured image stand out by compositing the inverted lowlight layer and the highlight layer.

Description

Technical Field

[0001] The present disclosure relates to an image processing method, an image processing system, and an image processing program.

Background Art

[0002] In Patent Document 1 below, when an autonomous underwater vehicle (hereinafter referred to as "AUV") docks with an underwater station, light is emitted from a light projecting unit of the underwater station, and the light functions as a marker indicating the navigation path of the AUV, and the AUV images the light (marker) with a light detection device.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, if a light source such as sunlight is included in the captured image taken by the light detection device of the AUV, the AUV may misidentify the light source as a marker. Such misidentification of the marker can occur not only in the AUV but also in other devices or operators that recognize the marker in the captured image.

[0005] Therefore, an object of the present disclosure is to provide an image processing method, an image processing system, and an image processing program for a captured image capable of suppressing misidentification of a marker.

Means for Solving the Problems

[0006] An image processing method according to one aspect of the present disclosure generates a lowlight layer from a captured image of a marker that emphasizes a portion of a dissimilar color that is not similar to the color of the marker, generates a highlight layer from the captured image that emphasizes a portion of a similar color that is the same as or similar to the color of the marker, generates an inverted lowlight layer by inverting the lowlight layer, and makes the portion of the marker in the captured image stand out by compositing the inverted lowlight layer and the highlight layer.

[0007] An image processing method according to another aspect of the present disclosure involves generating a lowlight layer from a captured image of a marker that emphasizes portions of dissimilar colors that are not similar to the color of the marker, generating a highlight layer from the captured image that emphasizes portions of similar colors that are the same as or similar to the color of the marker, generating inverted highlights by inverting the highlight layer, and highlighting the portion of the marker in the captured image by adding the inverted highlight layer and the lowlight layer and inverting them.

[0008] An image processing method according to another aspect of the present disclosure involves generating a plurality of lowlight layers from a captured image of a marker, emphasizing portions of dissimilar colors that are not similar to the color of the marker; generating a plurality of highlight layers from the captured image, emphasizing portions of similar colors that are the same as or similar to the color of the marker; generating a plurality of inverted lowlight layers by inverting the plurality of lowlight layers; and highlighting the portion of the marker in the captured image by compositing the plurality of inverted lowlight layers and the plurality of highlight layers.

[0009] An image processing method according to another aspect of the present disclosure involves generating a plurality of lowlight layers from a captured image of a marker, emphasizing portions of dissimilar colors that are not similar to the color of the marker; generating a plurality of highlight layers from the captured image, emphasizing portions of similar colors that are the same as or similar to the color of the marker; generating a plurality of inverted highlight layers by inverting the plurality of highlight layers; and highlighting the portion of the marker in the captured image by adding the plurality of inverted highlight layers and the plurality of lowlight layers and inverting them.

[0010] An image processing method according to another aspect of the present disclosure involves generating a lowlight layer from a captured image of a marker, highlighting a portion of the marker that is not similar in color to the marker; generating an inverted lowlight layer by inverting the lowlight layer; and highlighting the portion of the marker in the captured image by compositing the inverted lowlight layer with the captured image.

[0011] An image processing system according to one aspect of the present disclosure includes a camera for photographing a marker, and a processing device that acquires a captured image from the camera, generates a low-light layer from the captured image that emphasizes portions of dissimilar colors that are not similar to the color of the marker, generates a highlight layer from the captured image that emphasizes portions of similar colors that are the same as or similar to the color of the marker, generates an inverted low-light layer by inverting the low-light layer, and makes the portion of the marker in the captured image stand out by compositing the inverted low-light layer and the highlight layer.

[0012] An image processing program according to one aspect of the present disclosure causes a processing device to acquire a captured image from a camera, generate a lowlight layer from the captured image that emphasizes portions of dissimilar colors that are not similar to the color of the marker, generate a highlight layer from the captured image that emphasizes portions of similar colors that are the same as or similar to the color of the marker, generate an inverted lowlight layer by inverting the lowlight layer, and make the portion of the marker in the captured image stand out by compositing the inverted lowlight layer and the highlight layer. [Effects of the Invention]

[0013] According to the above configuration, it is possible to provide an image processing method, image processing system, and image processing program for captured images that can suppress misidentification of markers. [Brief explanation of the drawing]

[0014] [Figure 1] Figure 1 is an overall diagram of the image processing system. [Figure 2] Figure 2 is a flowchart illustrating the selection method for reference similar and reference dissimilar colors. [Figure 3] Figure 3(a) is an illustrative diagram of similar and dissimilar hue ranges, Figure 3(b) is an illustrative diagram of similar and dissimilar lightness ranges, and Figure 3(c) is an illustrative diagram of similar and dissimilar saturation ranges. [Figure 4] Figure 4 is a flowchart of the image processing program. [Figure 5] Figure 5 is a flowchart illustrating the selection method for multiple reference similar colors and multiple reference dissimilar colors related to Modification Example 1. [Figure 6] Figure 6 is an illustrative diagram of the similar and dissimilar hue ranges related to Modification 1. [Figure 7] Figure 7 is a flowchart of the image processing program related to Modification Example 3. [Modes for carrying out the invention]

[0015] (Overall configuration of the image processing system) Hereinafter, embodiments will be described. First, the overall configuration of the image processing system 100 will be described. FIG. 1 is an overall view of the image processing system 100. As shown in FIG. 1, the image processing system 100 according to the present embodiment includes a camera 10 and a processing device 20.

[0016] The camera 10 is a device for photographing the marker 101. The marker 101 may be a light emitter that emits light of a specific color, or may be an object having a color different from the surroundings. The camera 10 is communicably connected to the processing device 20, and can supply an image (hereinafter referred to as a "captured image") captured by the camera 10 to the processing device 20 as image data. The captured image of the present embodiment is a color image and is composed of a plurality of pixels arranged in a matrix. Further, the image data of the captured image includes the position (coordinates) and color components of each pixel.

[0017] In the present embodiment, the color components are expressed using RGB color values obtained by quantifying the intensities of red (R), green (G), and blue (B). Further, in the present embodiment, the lower limit values of R, G, and B are each set to 0, and the upper limit values are each set to 1. For example, the color components of a color with an R intensity of 0.2, a G intensity of 0.7, and a B intensity of 0.4 are expressed as (0.2, 0.7, 0.4) by arranging them in this order. However, the method of expressing the color components is not limited to the above, and for example, the lower limit values of R, G, and B may be set to 0 and the upper limit values may be set to 255 for expression.

[0018] Generally, color is composed of three elements: "hue", "brightness", and "saturation". Hue refers to the difference in color tone such as red, blue, and yellow, and a so-called hue circle is an arrangement of changing hues in a circular shape. Further, brightness is the degree of lightness of a color, and saturation is the degree of vividness of a color. These "hue", "brightness", and "saturation" can be expressed by the above color components.

[0019] The processing device 20 is a device that processes captured images. The processing device 20 has a processor, a volatile memory, a non-volatile memory, an I / O interface, etc. Various programs including an image processing program described later are stored in the non-volatile memory of the processing device 20, and the processor performs arithmetic processing using the volatile memory based on each program. The processing device 20 is communicably connected to the camera 10 as described above, and can acquire a captured image from the camera 10 as image data.

[0020] (Method for Selecting Reference Similar Color and Reference Dissimilar Color) Next, a method for selecting a reference similar color and a reference dissimilar color will be described. The reference similar color and the reference dissimilar color are used in an image processing program described later. The reference similar color is the same as or similar to the color of the marker 101 (the color of the light emitted by the marker 101 or the color of the marker 101 itself), and the reference dissimilar color is a color not similar to the color of the marker 101. It is assumed that the color of the marker 101 is known at the time of selecting the reference similar color and the reference dissimilar color.

[0021] Also, in the present embodiment, the entity that selects the reference similar color and the reference dissimilar color is the setter. The setter is, for example, a user or a manufacturer of the image processing system 100. However, instead of the setter, the processing device 20 may select the reference similar color and the reference dissimilar color.

[0022] Figure 2 is a flowchart of a method for selecting a reference similar color and a reference dissimilar color. As shown in Figure 2, first, the setter determines a similar hue range, a similar lightness range, and a similar chroma range (step S1).

[0023] Figure 3(a) is an image diagram of a similar hue range and a dissimilar hue range. The ring in Figure 3(a) shows a hue circle, and the circumferential position of the thick line indicates the hue of the color of the marker 101. In the present embodiment, as shown in Figure 3(a), the range including the hue of the color of the marker 101 (the white area) among the entire range of hues is determined as the similar hue range.

[0024] Figure 3(b) is an illustrative diagram of similar and dissimilar brightness ranges. In Figure 3(b), the position along the longitudinal direction of the band indicates brightness, and the position of the thick line indicates the brightness of the color of marker 101. In this embodiment, as shown in Figure 3(b), the range that includes the brightness of the color of marker 101 (the white-out area) within the entire range of brightness is determined as the similar brightness range.

[0025] Figure 3(c) is an illustrative diagram of similar and dissimilar saturation ranges. In Figure 3(c), the position along the longitudinal direction of the band indicates saturation, and the position of the thick line indicates the saturation of marker 101. In this embodiment, as shown in Figure 3(c), the range that includes the saturation of the color of marker 101 (the white-out area) within the entire range of saturation is determined as the similar saturation range.

[0026] The size of the similar hue range, similar lightness range, and similar saturation range is not limited. However, the similar hue range may be 10-50% of the entire hue range, the similar lightness range may be 10-50% of the entire lightness range, and the similar saturation range may be 10-50% of the entire saturation range.

[0027] Next, the setter determines the dissimilar hue range, dissimilar lightness range, and dissimilar saturation range (step S2). In this embodiment, as shown in Figure 3(a), the range of the entire hue range other than the similar hue range (shaded area) is determined to be the dissimilar hue range. Also, as shown in Figure 3(b), the range of the entire lightness range other than the similar lightness range (shaded area) is determined to be the dissimilar lightness range. Furthermore, as shown in Figure 3(c), the range of the entire saturation range other than the similar saturation range (shaded area) is determined to be the dissimilar saturation range.

[0028] However, a portion of the entire range of hue other than the similar hue range may be designated as the dissimilar hue range, a portion of the entire range of lightness other than the similar lightness range may be designated as the dissimilar lightness range, and a portion of the entire range of saturation other than the similar saturation range may be designated as the dissimilar saturation range. In this case, the dissimilar hue range may be 10-50% of the entire range of hue, the dissimilar lightness range may be 10-50% of the entire range of lightness, and the dissimilar saturation range may be 10-50% of the entire range of saturation.

[0029] Next, the user selects a reference similar color (step S3). In this embodiment, the reference similar color is selected from colors whose hue falls within the similar hue range described above, whose lightness falls within the similar lightness range described above, and whose saturation falls within the similar saturation range described above. The reference similar color may be the same color as the marker 101. The reference similar color is also assumed to have color components represented by RGB color values.

[0030] Next, the user selects a reference dissimilar color (step S4). In this embodiment, the reference dissimilar color is selected from colors whose hue falls within the aforementioned dissimilar hue range, whose lightness falls within the aforementioned dissimilar lightness range, or whose saturation falls within the aforementioned dissimilar saturation range. In other words, a non-reference similar color is selected from colors whose hue does not fall within the aforementioned similar hue range, whose lightness does not fall within the aforementioned similar lightness range, or whose saturation does not fall within the aforementioned similar saturation range.

[0031] However, the reference dissimilar color may be selected from a narrower range than the one described above. For example, the reference dissimilar color may be selected from colors whose hue falls within the aforementioned dissimilar hue range, whose lightness falls within the aforementioned dissimilar lightness range, and whose saturation falls within the aforementioned dissimilar saturation range. Furthermore, the reference dissimilar color is assumed to have color components represented by RGB color values.

[0032] In this embodiment, the reference similar color and reference dissimilar color were selected based on hue, lightness, and saturation, but the reference similar color and reference dissimilar color may also be selected based on hue alone. Specifically, the reference similar color may be selected from colors whose hue falls within the similar hue range described above, and the reference dissimilar color may be selected from colors whose hue falls within the dissimilar hue range described above. In this case, the reference similar color and reference dissimilar color may or may not have similar lightness and saturation. The above is a description of the method for selecting the reference similar color and reference dissimilar color in this embodiment.

[0033] (Image processing program) Next, the image processing program executed by the processing unit 20 will be described. Figure 4 is a flowchart of the image processing program. The image processing program in this embodiment is a program for highlighting the portion of the captured image where the marker 101 is located. It is assumed that the aforementioned reference similar color and reference dissimilar color have been selected in advance when the image processing program is executed.

[0034] When the image processing program starts, the processing unit 20 acquires the captured image from the camera 10 (step S11). As described above, the processing unit 20 can acquire the captured image from the camera 10 as image data. It is assumed that the captured image includes the marker 101.

[0035] Next, the processing unit 20 generates a lowlight layer (step S12). The lowlight layer is image data that highlights parts of the captured image that are not similar in color to the color of the marker 101 (hereinafter referred to as "dissimilar colors"). In the lowlight layer, a "lowlight value" is assigned to each pixel instead of a color component. The lowlight value assigned to each pixel is a value obtained by the dot product of the color component of that pixel in the captured image and the color component of the aforementioned reference dissimilar color. In other words, a lowlight layer can be generated by assigning the lowlight value obtained by the dot product of the color component of the pixel and the color component of the reference dissimilar color to the coordinates of each pixel that makes up the captured image.

[0036] For example, if the color components of a pixel at a certain coordinate in a captured image are (r, g, b) and the color components of a reference dissimilar color are (Ra, Ga, Ba), then the lowlight value, which is the dot product of the color components of that pixel and the color components of the reference dissimilar color, can be calculated as r × Ra + g × Ga + b × Ba. In step S12, this calculation is performed for all pixels, and each lowlight value obtained from the calculation is assigned to the coordinate of the corresponding pixel. This allows a lowlight layer to be generated. The lowlight layer can be visualized by representing the lowlight value of each pixel, for example, with shades of monochrome. This type of layer visualization can also be performed for each layer described later.

[0037] Next, the processing unit 20 generates a highlight layer (step S13). The highlight layer is image data in which the parts of the captured image that are the same as or similar in color to the marker 101 (hereinafter referred to as "similar color") are emphasized. In the highlight layer, a "highlight value" is assigned to each pixel instead of a color component. The highlight value assigned to each pixel is a value obtained by the dot product of the color component of that pixel in the captured image and the color component of the aforementioned reference similar color. In other words, a highlight layer can be generated by assigning the highlight value obtained by the dot product of the color component of the pixel and the color component of the reference similar color to the coordinates of each pixel that makes up the captured image.

[0038] For example, if the color components of a pixel at a certain coordinate in a captured image are (r, g, b) and the color components of a reference analogue are (Rb, Gb, Bb), then the highlight value, which is the dot product of the color components of that pixel and the color components of the reference analogue, can be calculated as r × Rb + g × Gb + b × Bb. In step S13, this calculation is performed for all pixels, and each highlight value obtained from the calculation is assigned to the coordinate of the corresponding pixel. This allows a lowlight layer to be generated.

[0039] Next, the processing unit 20 generates an inverted lowlight layer (step S14). An inverted lowlight layer is image data obtained by inverting the lowlight layer. In this embodiment, an inverted lowlight layer is generated by assigning the "inverted lowlight value" of each pixel to the coordinates of each pixel that constitutes the lowlight layer generated in step S12.

[0040] The term "inverted lowlight value" as used here refers to the value obtained by inverting the lowlight value, and is the symmetrical value of the lowlight value relative to a predetermined intermediate value. For example, if the intermediate value is "1.5" and the lowlight value is "1", then the inverted lowlight value is "2", which is the symmetrical value of the lowlight value relative to the intermediate value. In other words, the inverted lowlight value is "2", which is obtained by adding the difference between the intermediate value and the lowlight value, "0.5", to the intermediate value of "1.5". Similarly, if the intermediate value is "1.5" and the lowlight value is "2.5", then the inverted lowlight value is "0.5". Note that the above intermediate value may be the value midway between the maximum and minimum values ​​of the lowlight value, or it may be any other value.

[0041] Next, the processing unit 20 combines the inverted lowlight layer and the highlight layer (step S15). In this embodiment, the inverted lowlight layer and the highlight layer are combined by multiplying them together. Specifically, the inverted lowlight value of the inverted lowlight layer and the highlight value of the highlight layer are multiplied by their corresponding pixels, and the resulting value is assigned to the coordinates of each pixel. By performing this process, image data combining the inverted lowlight layer and the highlight layer can be generated.

[0042] However, instead of multiplying the inverted lowlight layer and the highlight layer, you can also combine the inverted lowlight layer and the highlight layer by adding them together. Note that multiplication of layers is the process of multiplying the numerical values ​​of the corresponding pixels and assigning the resulting value to the corresponding pixels. Similarly, addition of layers is the process of adding the numerical values ​​of the corresponding pixels and assigning the resulting value to the corresponding pixels. Both layer multiplication and layer addition are equivalent to layer compositing.

[0043] The above describes the image processing program according to this embodiment. As described above, the image processing program according to this embodiment combines an inverted lowlight layer with a highlight layer. This process enhances the parts of the captured image that are similar in color to the marker 101, and further suppresses the parts of the captured image that are not similar in color to the marker 101. Therefore, the above image processing program makes it possible to obtain image data in which the part of the captured image that is the marker 101 stands out, and to suppress misidentification of the marker 101 in the captured image.

[0044] (Variation 1) In the embodiment described above, one reference analogue and one reference dissimilar color are selected, one lowlight layer, one highlight layer, and one inverted lowlight layer are generated, and then one inverted lowlight layer and one highlight layer are combined. However, the number of reference analogues and reference dissimilar colors to be selected and the number of each layer to be generated are not limited. For example, multiple reference analogues and reference dissimilar colors may be selected, multiple lowlight layers, highlight layers, and inverted lowlight layers may be generated, and then multiple inverted lowlight layers and multiple highlight layers may be combined.

[0045] Below, as Modification 1, we will describe an example of a method in which multiple reference similar colors and multiple reference dissimilar colors are selected and an image processing program is executed using these.

[0046] Here, based on "hue," one of the three elements of color—"hue," "lightness," and "saturation"—multiple reference similar colors and multiple reference dissimilar colors are selected. Figure 5 is a flowchart of the method for selecting multiple reference similar colors and multiple reference dissimilar colors according to Modification Example 1. As shown in Figure 5, in the method for selecting multiple reference similar colors and multiple reference dissimilar colors according to Modification Example 1, the setter first determines the range of similar hues (step S21).

[0047] Here, the rings in Figures 6(a) and 6(b) represent the hue wheel, and the circumferential position of the thick line in Figure 6(a) indicates the hue of the color of marker 101. In this modified example, as shown in Figure 6(a), the range that includes the hue of the color of marker 101 (the white-outlined area) within the entire range of hues is determined as the similar hue range. The size of the similar hue range is not limited. However, the similar hue range may be 10-50% of the entire range of hues.

[0048] Next, the setter determines the range of dissimilar hues (step S22). In this modified example, as shown in Figure 6(a), the range of the entire hue that is not the range of similar hues (the shaded area) is determined to be the range of dissimilar hues. However, a portion of the range of the entire hue that is not the range of similar hues may be determined to be the range of dissimilar hues. In this case, the range of dissimilar hues may be 10 to 50% of the entire hue range.

[0049] Next, the setter divides the similar hue range into multiple partial similar hue ranges (step S23). In Figure 6(b), the dashed lines within the similar hue range indicate the boundaries of the partial similar hue ranges. In this modified example, as shown in Figure 6(b), the similar hue range is divided into two partial similar hue ranges. Furthermore, in this modified example, the similar hue range is divided into two partial similar hue ranges such that the size of each partial similar hue range is equal. However, the size of each partial similar hue range is not limited.

[0050] Next, the setter divides the dissimilar hue range into multiple partial dissimilar hue ranges (step S24). In Figure 6(b), the dashed lines within the dissimilar hue range indicate the boundaries of the partial dissimilar hue ranges. In this modified example, as shown in Figure 6(b), the dissimilar hue range is divided into four partial dissimilar hue ranges. Furthermore, in this modified example, the dissimilar hue range is divided into four partial similar hue ranges such that the size of each partial dissimilar hue range is equal. Moreover, the size of each partial dissimilar hue range is equal to the size of each partial similar hue range described above. However, the size of each partial similar hue range is not limited.

[0051] Next, the user selects a reference analogue (step S25). In this modified example, for each partial analogue range, a reference analogue is selected from the colors of the hues included in that partial analogue range. In other words, in this modified example, two reference analogues are selected.

[0052] Next, the user selects a reference dissimilar color (step S26). In this modified example, for each partial dissimilar hue range, a reference dissimilar color is selected from the hues included in that partial dissimilar hue range. In other words, in this modified example, four reference dissimilar colors are selected. This concludes the explanation of how to select multiple reference similar colors and multiple reference dissimilar colors.

[0053] As described above, after selecting multiple reference similar colors and multiple reference dissimilar colors, in this modified example, the image processing program is executed using the multiple reference similar colors and multiple reference dissimilar colors. The image processing program in this modified example is the same as the image processing program of the embodiment described above (Figure 4).

[0054] Specifically, the processing unit 20 first acquires the captured image from the camera 10 (corresponding to step S11 in Figure 4). Subsequently, the processing unit 20 generates multiple low-light layers (corresponding to step S12). In this modified example, the processing unit 20 performs a process for each reference dissimilar color to assign a low-light value obtained by the dot product of the color component of the pixel and the color component of the reference dissimilar color to the coordinates of each pixel constituting the captured image. This allows for the generation of multiple low-light layers.

[0055] Next, the processing unit 20 generates multiple highlight layers (corresponding to step S13). In this modified example, the processing unit 20 performs a process for each reference similar color to assign a highlight value obtained by the dot product of the color component of the pixel and the color component of the reference similar color to the coordinates of each pixel constituting the captured image. This allows for the generation of multiple highlight layers.

[0056] Next, the processing unit 20 inverts each lowlight layer to generate multiple inverted lowlight layers (corresponding to step S14). Then, the processing unit 20 combines the multiple inverted lowlight layers and multiple highlight layers (corresponding to step S15).

[0057] The compositing of multiple inverted lowlight layers and multiple highlight layers may be performed as follows: First, one of the multiple inverted lowlight layers is multiplied sequentially with multiple highlight layers to generate multiple primary layers, and then all of the generated primary layers are multiplied together to generate a secondary layer. This is done for each inverted lowlight layer to generate multiple secondary layers, and then all of the generated secondary layers are added together. This allows multiple inverted lowlight layers and multiple highlight layers to be composited. However, multiple inverted lowlight layers and multiple highlight layers may be composited using methods other than those described above.

[0058] The above processing uses multiple reference similar colors and multiple reference dissimilar colors to emphasize the similar color portions of marker 101 in the captured image, and further suppresses the dissimilar color portions of marker 101. Therefore, by implementing the above image processing program, it is possible to obtain image data in which the portion of marker 101 in the captured image is highlighted, and misidentification of marker 101 in the captured image can be suppressed.

[0059] (Modification 2) In the embodiment described above, in step S15 of the image processing program, the processing unit 20 combined the inverted lowlight layer and the highlight layer, but the inverted lowlight layer and the captured image may also be combined. Specifically, the inverted lowlight values ​​of the inverted lowlight layer and the color components of the captured image may be multiplied together for corresponding pixels and added, and the resulting values ​​may be assigned to the coordinates of each pixel.

[0060] For example, if the color components of a pixel at a certain coordinate in the captured image are (r, g, b), and the inverted lowlight value of the corresponding pixel in the inverted lowlight layer is "X", then the value calculated as r × X + g × X + b × X is assigned to the coordinate of the corresponding pixel. By performing this process for all pixels, the inverted lowlight layer and the captured image can be combined. Even with this method, it is possible to obtain image data that highlights the portion of marker 101 in the captured image, thereby suppressing misidentification of marker 101 in the captured image.

[0061] (Variation 3) In the image processing program of the embodiment described above, the processing unit 20 generates an inverted lowlight layer (step S14) and multiplies the inverted lowlight layer and the highlight layer (step S15). However, the processing unit 20 may generate an inverted highlight layer instead of an inverted lowlight layer and invert the inverted highlight layer and the lowlight layer by adding them together. Below, as Modification 3, an image processing program that performs this method will be described.

[0062] Figure 7 is a flowchart of the image processing program according to Modification 3. When the image processing program according to Modification 3 is started, the processing unit 20 acquires the captured image from the camera 10 (step S31), generates a low-light layer (step S32), and generates a highlight layer (step S33). These steps S31 to S33 are the same as steps S11 to S13 in the embodiment described above (Figure 4).

[0063] Next, the processing unit 20 generates an inverted highlight layer (step S34). The inverted highlight layer can be generated by inverting the highlight layer. Specifically, the inverted highlight layer is generated by assigning an "inverted highlight value," which is the inverted highlight value of a pixel, to the coordinates of each pixel that makes up the highlight layer. The method for calculating the inverted highlight value is the same as the method for calculating the inverted low light value.

[0064] Next, the inverted highlight layer and the lowlight layer are combined (step S35). In this modified example, the combination of the inverted highlight layer and the lowlight layer is performed by adding the inverted highlight layer and the lowlight layer. Specifically, the inverted highlight values ​​of the inverted highlight layer and the lowlight values ​​of the lowlight layer are added together for corresponding pixels, and the resulting value is assigned to the coordinates of each pixel. By performing this process, a "composite layer," which is a combined layer of the inverted highlight layer and the lowlight layer, can be generated.

[0065] Next, in step S35, the composite layer is inverted (step S36). This generates an image with the composite layer inverted. Even with this method, it is possible to obtain image data in which the portion of marker 101 in the captured image is highlighted. As a result, misidentification of marker 101 in the captured image can be suppressed.

[0066] (Modification 4) The above describes each variation, but these variations may be combined. For example, Variation 1 and Variation 3 may be combined. Specifically, as described in Variation 1, the processing device 20 generates multiple highlight layers and multiple lowlight layers. Then, similar to Variation 3, the multiple highlight layers are inverted to generate multiple inverted highlight layers (see step S34), the multiple inverted highlight layers and multiple lowlight layers are combined to generate a composite layer (step S35), and the composite layer may be inverted again (see step S36). Even with this method, image data in which the portion of marker 101 in the captured image is highlighted can be obtained. As a result, misidentification of marker 101 in the captured image can be suppressed.

[0067] The functions of the elements disclosed herein can be performed using circuits or processing circuits, including general-purpose processors, dedicated processors, integrated circuits, ASICs (Application Specific Integrated Circuits), conventional circuits, and / or combinations thereof, configured or programmed to perform the disclosed functions. A processor is considered a processing circuit or circuit because it includes transistors and other circuits. In this disclosure, a circuit, unit, or means is hardware that performs the enumerated functions, or hardware programmed to perform the enumerated functions. The hardware may be hardware disclosed herein, or other known hardware that is programmed or configured to perform the enumerated functions. If the hardware is a processor, which is considered a type of circuit, then the circuit, means, or unit is a combination of hardware and software, and the software is used to configure the hardware and / or the processor.

[0068] (summary) The first item disclosed herein is an image processing method that generates a lowlight layer from a captured image of a marker, emphasizing portions of dissimilar colors that are not similar to the color of the marker; generates a highlight layer from the captured image, emphasizing portions of similar colors that are the same as or similar to the color of the marker; generates an inverted lowlight layer by inverting the lowlight layer; and makes the portion of the marker in the captured image stand out by compositing the inverted lowlight layer and the highlight layer.

[0069] This method enhances the areas of the marker that are similar in color to the marker within the captured image, while suppressing areas that are different in color. As a result, it is possible to obtain image data that highlights the marker within the captured image.

[0070] A second item disclosed herein is the image processing method described in the first item, which generates the lowlight layer by assigning to the coordinates of each pixel constituting the captured image a value obtained by the dot product of the color component of the pixel and the color component of a reference dissimilar color that is not similar to the color of the marker.

[0071] This method allows for the creation of a low-light layer from a captured image that emphasizes areas with dissimilar colors that do not resemble the marker color.

[0072] A third item disclosed herein is the image processing method described in the second item, which generates the highlight layer by assigning to the coordinates of each pixel constituting the captured image a value obtained by the dot product of the color component of the pixel and the color component of a reference similar color that is the same as or similar to the color of the marker.

[0073] This method allows you to generate a highlight layer from the captured image that emphasizes areas with similar or identical colors to the marker color.

[0074] The fourth item disclosed herein is an image processing method according to the third item, which involves determining a similar hue range including the hue of the marker's color, a similar lightness range including the lightness of the marker's color, and a similar saturation range including the saturation of the marker's color; selecting the reference similar color from colors whose hue is included in the similar hue range, whose lightness is included in the similar lightness range, and whose saturation is included in the similar saturation range; and selecting the reference dissimilar color from colors whose hue is not included in the similar hue range, whose lightness is not included in the similar lightness range, or whose saturation is not included in the similar saturation range.

[0075] This method allows for the selection of appropriate reference similarities and reference dissimilarities, thereby enabling the generation of appropriate highlight and lowlight layers.

[0076] The fifth item disclosed herein is an image processing method according to any one of the first to fourth items, wherein the compositing of the inverted lowlight layer and the highlight layer is a multiplication of the inverted lowlight layer and the highlight layer.

[0077] Even with this method, it is possible to obtain image data in which the marker portion within the captured image is highlighted.

[0078] The sixth item disclosed herein is an image processing method that generates a low-light layer from a captured image of a marker, emphasizing portions of dissimilar colors that are not similar to the color of the marker; generates a highlight layer from the captured image, emphasizing portions of similar colors that are the same as or similar to the color of the marker; generates an inverted highlight layer by inverting the highlight layer; and makes the portion of the marker in the captured image stand out by adding the inverted highlight layer and the low-light layer and inverting them.

[0079] This method inverts the image, highlighting areas other than the marker within the captured image. Therefore, even with this method, it is possible to obtain image data that highlights the marker within the captured image, thus suppressing misidentification of the marker in the captured image.

[0080] The seventh item disclosed herein is an image processing method that generates a plurality of lowlight layers from a captured image of a marker, emphasizing portions of dissimilar colors that are not similar to the color of the marker; generates a plurality of highlight layers from the captured image, emphasizing portions of similar colors that are the same as or similar to the color of the marker; generates a plurality of inverted lowlight layers by inverting the plurality of lowlight layers; and makes the portion of the marker in the captured image stand out by compositing the plurality of inverted lowlight layers and the plurality of highlight layers.

[0081] This method utilizes multiple lowlight and highlight layers, allowing for further emphasis on the markers within the captured image. As a result, misidentification of markers within the image can be significantly reduced.

[0082] The eighth item disclosed herein is an image processing method according to the seventh item, which generates a plurality of lowlight layers by performing a calculation on a plurality of reference dissimilar colors to assign to the coordinates of each pixel constituting the captured image a value obtained by the dot product of the color component of the pixel and the color component of a reference dissimilar color that is not similar to the color of the marker, and generates a plurality of highlight layers by performing a calculation on a plurality of reference similar colors to assign to the coordinates of each pixel constituting the captured image a value obtained by the dot product of the color component of the pixel and the color component of a reference similar color that is the same as or similar to the color of the marker.

[0083] This method allows for the generation of multiple lowlight layers from a captured image that emphasize areas with dissimilar colors that are not similar to the marker color, as well as multiple highlight layers from the captured image that emphasize areas with similar colors that are the same as or similar to the marker color.

[0084] The ninth item disclosed herein is an image processing method according to the eighth item, which involves determining a similar hue range including the hue of the marker's color, and a dissimilar hue range other than the similar hue range, selecting the plurality of reference similar colors from among the colors whose hue is included in the similar hue range, and selecting the plurality of reference dissimilar colors from among the colors whose hue is included in the dissimilar hue range.

[0085] This method allows for the selection of multiple reference similarities and multiple reference dissimilarities, thereby enabling the generation of multiple highlight layers and multiple lowlight layers.

[0086] The tenth item disclosed herein is the image processing method described in item 9, which divides the similar hue range into a plurality of partial similar hue ranges, divides the dissimilar hue range into a plurality of partial dissimilar hue ranges, selects a reference similar color for each partial similar hue range, and selects a reference dissimilar color for each partial dissimilar hue range.

[0087] This method allows for the selection of multiple appropriate reference similarities and multiple reference dissimilarities, thereby enabling the generation of multiple appropriate highlight and lowlight layers.

[0088] The eleventh item disclosed herein is an image processing method that generates a plurality of lowlight layers from a captured image of a marker, emphasizing portions of dissimilar colors that are not similar to the color of the marker; generates a plurality of highlight layers from the captured image, emphasizing portions of similar colors that are the same as or similar to the color of the marker; generates a plurality of inverted highlight layers by inverting the plurality of highlight layers; and makes the portion of the marker in the captured image stand out by adding the plurality of inverted highlight layers and the plurality of lowlight layers and inverting them.

[0089] Even with this method, it is possible to obtain image data in which the marker portion within the captured image is highlighted, thereby suppressing misidentification of the marker within the captured image.

[0090] The twelfth item disclosed herein is an image processing method that generates a lowlight layer from a captured image of a marker, highlighting a portion of the marker that is not similar in color to the marker; generates an inverted lowlight layer by inverting the lowlight layer; and highlights the portion of the marker in the captured image by compositing the inverted lowlight layer with the captured image.

[0091] Even with this method, by suppressing areas of dissimilar color to the marker in the captured image, it is possible to obtain image data in which the marker portion stands out within the captured image. As a result, misidentification of the marker within the captured image can be suppressed.

[0092] A thirteenth item disclosed herein is an image processing system comprising: a camera for photographing a marker; an image processing device that acquires an image from the camera, generates a lowlight layer from the image that emphasizes portions of dissimilar color that are not similar to the color of the marker, generates a highlight layer from the image that emphasizes portions of similar color that are the same as or similar to the color of the marker, generates an inverted lowlight layer by inverting the lowlight layer, and enhances the portion of the marker in the image by compositing the inverted lowlight layer and the highlight layer.

[0093] This configuration emphasizes the parts of the captured image that are similar in color to the marker, while suppressing the parts that are not similar in color, thereby obtaining image data that highlights the marker in the captured image. As a result, misidentification of the marker in the captured image can be suppressed.

[0094] The fourteenth item disclosed herein is an image processing program for causing a processing device to acquire a captured image from a camera, generate a lowlight layer from the captured image that emphasizes portions of dissimilar colors that are not similar to the color of a marker, generate a highlight layer from the captured image that emphasizes portions of similar colors that are the same as or similar to the color of the marker, generate an inverted lowlight layer by inverting the lowlight layer, and combine the inverted lowlight layer and the highlight layer to highlight the portion of the marker in the captured image.

[0095] This configuration emphasizes the parts of the captured image that are similar in color to the marker, while suppressing the parts that are not similar in color, thereby obtaining image data that highlights the marker in the captured image. As a result, misidentification of the marker in the captured image can be suppressed. [Explanation of Symbols]

[0096] 10 Cameras 20 Processing Units 100 Image Processing Systems 101 Markers

Claims

1. An image processing method that generates a lowlight layer from a captured image of a marker, emphasizing the parts of the image that are dissimilar in color to the marker; generates a highlight layer from the captured image, emphasizing the parts of the image that are the same as or similar in color to the marker; generates an inverted lowlight layer by inverting the lowlight layer; and makes the parts of the image of the marker stand out by compositing the inverted lowlight layer and the highlight layer.

2. The image processing method according to claim 1, wherein the lowlight layer is generated by assigning to the coordinates of each pixel constituting the captured image a value obtained by the dot product of the color component of the pixel and the color component of a reference dissimilar color that is not similar to the color of the marker.

3. The image processing method according to claim 2, wherein the highlight layer is generated by assigning to the coordinates of each pixel constituting the captured image a value obtained by the dot product of the color component of the pixel and the color component of a reference similar color that is the same as or similar to the color of the marker.

4. Determine the similar hue range including the hue of the marker's color, the similar lightness range including the lightness of the marker's color, and the similar saturation range including the saturation of the marker's color. From among colors whose hue falls within the similar hue range, whose lightness falls within the similar lightness range, and whose saturation falls within the similar saturation range, the reference similar color is selected. The image processing method according to claim 3, wherein the reference dissimilar color is selected from colors whose hue is not included in the similar hue range, whose lightness is not included in the similar lightness range, or whose saturation is not included in the similar saturation range.

5. The image processing method according to claim 1, wherein the compositing of the inverted lowlight layer and the highlight layer is a multiplication of the inverted lowlight layer and the highlight layer.

6. An image processing method that generates a lowlight layer from a captured image of a marker, emphasizing the parts of the image that are dissimilar in color to the marker; generates a highlight layer from the captured image, emphasizing the parts of the image that are the same as or similar in color to the marker; generates an inverted highlight layer by inverting the highlight layer; and makes the parts of the image of the marker stand out by adding the inverted highlight layer and the lowlight layer and inverting them.

7. An image processing method that generates multiple lowlight layers from a captured image of a marker, emphasizing areas of dissimilar color that are not similar to the color of the marker; generates multiple highlight layers from the captured image, emphasizing areas of similar color that are the same as or similar to the color of the marker; generates multiple inverted lowlight layers by inverting the multiple lowlight layers; and makes the portion of the marker in the captured image stand out by compositing the multiple inverted lowlight layers and the multiple highlight layers.

8. By performing a calculation on multiple reference dissimilar colors to assign to the coordinates of each pixel constituting the captured image a value obtained by the dot product of the color component of that pixel and the color component of a reference dissimilar color that is not similar to the color of the marker, the multiple lowlight layers are generated. The image processing method according to claim 7, wherein a plurality of highlight layers are generated by performing a calculation on a plurality of reference similar colors to assign to the coordinates of each pixel constituting the captured image a value obtained by the dot product of the color component of the pixel and the color component of a reference similar color that is the same as or similar to the color of the marker.

9. Determine the range of similar hues that includes the hue of the marker's color, and the range of dissimilar hues that are outside the range of similar hues. Select the multiple reference similar colors from among the colors whose hue falls within the aforementioned similar hue range. The image processing method according to claim 8, wherein a plurality of reference dissimilar colors are selected from among colors whose hue falls within the dissimilar hue range.

10. The aforementioned similar hue range is divided into multiple partially similar hue ranges, The aforementioned dissimilar hue range is divided into multiple partially dissimilar hue ranges, Select a reference similar color for each partial similar hue range, The image processing method according to claim 9, wherein a reference dissimilar color is selected for each partial dissimilar hue range.

11. An image processing method that generates multiple lowlight layers from a captured image of a marker, emphasizing areas of dissimilar color that are not similar to the color of the marker; generates multiple highlight layers from the captured image, emphasizing areas of similar color that are the same as or similar to the color of the marker; generates multiple inverted highlight layers by inverting the multiple highlight layers; and makes the portion of the marker in the captured image stand out by adding the multiple inverted highlight layers and the multiple lowlight layers and inverting them.

12. An image processing method that generates a lowlight layer from a captured image of a marker, highlighting the portion of the marker that is a dissimilar color to the color of the marker; generates an inverted lowlight layer by inverting the lowlight layer; and highlights the portion of the marker in the captured image by compositing the inverted lowlight layer with the captured image.

13. A camera to photograph the marker, Image processing system comprising: an image capture device that acquires an image from the camera; generates a lowlight layer from the image that emphasizes portions of the image that are dissimilar in color to the marker; generates a highlight layer from the image that emphasizes portions of the image that are the same as or similar in color to the marker; generates an inverted lowlight layer by inverting the lowlight layer; and combines the inverted lowlight layer and the highlight layer to make the portion of the image that contains the marker stand out.

14. An image processing program for making the portion of the marker stand out in the captured image by causing a processing device to acquire a captured image from a camera, generating a lowlight layer from the captured image that emphasizes portions of dissimilar colors that are not similar to the color of the marker, generating a highlight layer from the captured image that emphasizes portions of similar colors that are the same as or similar to the color of the marker, generating an inverted lowlight layer by inverting the lowlight layer, and compositing the inverted lowlight layer and the highlight layer.

Images