Detection device, detection method, and program
By combining models with different detection characteristics and using height-based determinations, the detection accuracy of image regions in grouped articles is enhanced, addressing the inconsistency in existing image detection technologies.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- NEC CORP
- Filing Date
- 2023-03-28
- Publication Date
- 2026-05-19
AI Technical Summary
Existing image detection models struggle with inconsistent detection accuracy for different types of image regions, particularly in identifying both foreground and background areas in images of grouped articles.
Employing a combination of first and second models with distinct detection characteristics to identify different image regions, where the first model focuses on the main occupied space of items and the second model detects items in the foreground, using average height calculations to determine which model to apply to specific image regions.
Improves the overall detection accuracy of image regions by compensating for the weaknesses of individual models, allowing for precise identification of both foreground and background items in captured images.
Smart Images
Figure 0007861912000001 
Figure 0007861912000002 
Figure 0007861912000003
Abstract
Description
Technical Field
[0001] The present disclosure relates to a detection device, a detection method, and a non - transient computer - readable medium.
Background Art
[0002] Techniques have been proposed for detecting regions where groups of articles, such as products, continuously exist in images of groups of articles using one learned identification model (for example, Patent Document 1).
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] The inventor has found that the detection accuracy of image regions may not be sufficiently obtained with one identification model. That is, generally, an identification model has objects that it is good at detecting and objects that it is not good at detecting. Therefore, the inventor has found that the detection accuracy of image regions can be improved by applying first and second models with different characteristics to an image of a group of articles.
[0005] One object of the present disclosure is to provide a detection device, a detection method, and a non - transient computer - readable medium that can improve the detection accuracy of image regions. It should be noted that this object is only one of the multiple objects that the multiple embodiments disclosed in this specification attempt to achieve. Other objects or problems and novel features will be clarified from the description of this specification or the attached drawings.
Means for Solving the Problems
[0006] In one aspect, the detection device is A first detection unit detects the second image region in a captured image of a shelf using the second model, which is one of a first model that identifies a first image region based on the main occupied space of an item group in a target image, and a second model that identifies a second image region corresponding to an image of an item group placed in the front of the target image. A selection unit that identifies multiple shelf image regions corresponding to multiple shelf levels of the item shelf in the captured image, An average value calculation unit that calculates the average value of the height of the second image region corresponding to the height direction of the item shelf in each shelf image region, A determination unit that determines the image region to be processed from among the plurality of shelf image regions based on the average value in each shelf image region and the height of each shelf image region, A second detection unit detects the first image region in the image region to be processed by applying the first model to the image region to be processed, It is equipped with.
[0007] In other embodiments, the detection method is: A first model identifies a first image region based on the main occupied space of an item group in the target image, and a second model identifies a second image region corresponding to an image of an item group placed in the foreground of the target image. Using the second model, the second image region in a captured image of an item shelf is detected. In the aforementioned captured image, a plurality of shelf image regions corresponding to each of the plurality of shelf levels of the item shelf are identified, To calculate the average height of the second image region corresponding to the height direction of the item shelf in each shelf image region, Based on the average value in each shelf image region and the height of each shelf image region, the image region to be processed is determined from among the plurality of shelf image regions. The first image region in the image region to be processed is detected by applying the first model to the image region to be processed, Includes.
[0008] In other embodiments, non-temporary computer-readable media are A first model identifies a first image region based on the main occupied space of an item group in the target image, and a second model identifies a second image region corresponding to an image of an item group placed in the foreground of the target image. Using the second model, the second image region in a captured image of an item shelf is detected. In the aforementioned captured image, a plurality of shelf image regions corresponding to each of the plurality of shelf levels of the item shelf are identified, To calculate the average height of the second image region corresponding to the height direction of the item shelf in each shelf image region, Based on the average value in each shelf image region and the height of each shelf image region, the image region to be processed is determined from among the plurality of shelf image regions. The first image region in the image region to be processed is detected by applying the first model to the image region to be processed, It stores a program that causes the detection device to execute a process that includes [specific details]. [Effects of the Invention]
[0009] This disclosure provides a detection device, a detection method, and a non-temporary computer-readable medium that can improve the detection accuracy of image regions. [Brief explanation of the drawing]
[0010] [Figure 1] This is a block diagram showing an example of a detection device in the first embodiment. [Figure 2] This flowchart shows an example of the processing operation of the detection device in the first embodiment. [Figure 3] This is a block diagram showing an example of a detection device in the second embodiment. [Figure 4A] This is a diagram showing an example of a shelf image. [Figure 4B] This figure shows an example of the second image region. [Figure 4C]It is a diagram showing an example of a first image area. [Figure 5] It is a block diagram showing an example of a detection device in the third embodiment. [Figure 6] It is a diagram showing an example of an integrated image. [Figure 7] It is a diagram showing an example of the hardware configuration of a detection device.
Embodiments for Carrying Out the Invention
[0011] Hereinafter, embodiments will be described with reference to the drawings. In the embodiments, the same or equivalent elements are denoted by the same reference numerals, and redundant descriptions are omitted.
[0012] <First Embodiment> <Overview of Detection Device> The detection device 10 in the first embodiment detects an image area corresponding to an image of an article group in a "captured image" using, for example, a "first model" and a "second model" having different detection characteristics from each other. The "captured image" is, for example, an image of an article shelf (hereinafter sometimes referred to as an "article shelf image").
[0013] The above "second model" is a model for identifying an image area (hereinafter sometimes referred to as a "second image area" or a "second article group image area") corresponding to an image of an article group arranged on the front side in a target image. Hereinafter, the "image of an article group" may sometimes be referred to as an "article group image".
[0014] For example, the second model may be a learned model learned using learning data including the following images. - An image in which the entire one side of a single article is captured. - An article shelf image in which an image area corresponding to an article group image of an article group arranged in the forefront row of each shelf tier in the article shelf image is designated as an "article group image area".
[0015] In addition, for items arranged in the front row of a shelf, typically a large portion of one side of the item (for example, more than half of one side) is visible in the shelf image.
[0016] When training is performed using this type of training data, the second model can accurately detect image regions corresponding to product groups located at the front of each shelf, but it may not be able to accurately detect image regions corresponding to product groups located behind the front-facing product groups where a large portion of one side is hidden.
[0017] Furthermore, the "first model" described above is a model that identifies an image region (hereinafter sometimes referred to as the "first image region" or "first group of items image region") based on the space primarily occupied by a group of items in the target image (hereinafter sometimes referred to as the "mainly occupied space of items").
[0018] For example, the first model may be a pre-trained model that has been trained using training data including the following images. - An image showing the entirety of one side of an object. - A shelf image in which the image region corresponding to the item group image of the item group arranged in the front row of each shelf is designated as the "item group image region". - A shelf image in which the image area corresponding to the "image equivalent to the true background (hereinafter sometimes referred to as the "true background image")" is designated as the "true background image area".
[0019] Here, the "true background image area" includes an image area corresponding to the image of the back panel, side panel, or shelf of the shelving unit that is not obscured by the shadows of the items in the shelving unit image.
[0020] Through training using such training data, the first model has the characteristic of being able to accurately detect image regions that are likely to be background images (for example, image regions corresponding to large empty spaces where no items are placed) and image regions corresponding to the above-mentioned spaces occupied by items. However, because the first model learns conflicting information such as "item group image regions" and "true background image regions," for regions that cannot be designated as either, there is a possibility that regions close to the "item group image region" will be detected as "item group image regions," and regions close to the "true background image region" will be detected as "true background image regions." Therefore, there is a possibility that an image region corresponding to a narrow empty space sandwiched between two "item group image regions" will be detected as part of the item group image region. This is because it is thought that in many cases, sufficient information to detect that an image is a background image cannot be obtained from an image corresponding to a narrow empty space.
[0021] Furthermore, unlike the training data for the first model, the training data for the second model described above does not include any shelf images with a specified "true background image region," or if it does, the number of such images is small.
[0022] As described above, the detection device 10 uses a "first model" and a "second model" with different detection characteristics to detect the image region corresponding to the image of the group of objects in the captured image. This improves the detection accuracy of the image region.
[0023] <Example of detection device configuration> Figure 1 is a block diagram showing an example of a detection device in the first embodiment. In Figure 1, the detection device 10 includes a detection unit (first detection unit) 11, a identification unit 12, an average value calculation unit 13, a detection unit (second detection unit) 14, and a determination unit 15. The detection device 10 acquires a captured image. This captured image is, for example, an image of a shelving unit. The following explanation assumes that the captured image is an image of a shelving unit. This shelving unit has multiple shelves.
[0024] The detection unit 11 detects the second image region in the captured image by applying the second model to the captured image.
[0025] The identification unit 12 identifies multiple "shelf image regions" in the captured image, each corresponding to a plurality of shelf levels. A shelf image region corresponding to one shelf level is, for example, an image region corresponding to the space between the shelf board of that shelf level and the shelf board of the shelf level immediately above it.
[0026] The average value calculation unit 13 calculates the average value of the height of the second image region in each shelf image region. The height of the second image region is the length of the second image region corresponding to the height direction of the item shelf.
[0027] The determination unit 15 determines the image area to be processed by the detection unit 14 from among a plurality of shelf image areas, based on the average value of the height of the second image area in each shelf image area and the height of each shelf image area. The height of the shelf image area is the length of the shelf image area corresponding to the height direction of the goods shelf.
[0028] The detection unit 14 detects the first image region in the image region to be processed by applying the first model to the image region to be processed determined by the determination unit 15.
[0029] <Example of detection device operation> An example of the processing operation of the detection device 10 having the above configuration will now be described. Figure 2 is a flowchart showing an example of the processing operation of the detection device in the first embodiment.
[0030] The detection unit 11 detects the second image region in the captured image by applying the second model to the captured image (step S101).
[0031] The identification unit 12 identifies multiple shelf image regions in the captured image that correspond to multiple shelf levels (step S102).
[0032] The average value calculation unit 13 calculates the average value of the height of the second image region in each shelf image region (step S103).
[0033] The determination unit 15 determines the image area to be processed by the detection unit 14 from among the multiple shelf image areas based on the average value of the height of the second image area in each shelf image area and the height of each shelf image area (step S104).
[0034] The detection unit 14 detects the first image region in the image region to be processed by applying the first model to the image region to be processed determined by the determination unit 15 (step S105).
[0035] As described above, according to the first embodiment, the detection device 10 uses a "first model" and a "second model" with different detection characteristics to detect the object group image region in the captured image. This allows one model to compensate for the other model's weakness in detecting certain objects, thereby improving the detection accuracy of the object group image region.
[0036] Furthermore, in the detection device 10, the detection unit 11 detects the second image region in the captured image by applying the second model to the captured image. The determination unit 15 determines the image region to be processed by the detection unit 14 from among a plurality of shelf image regions based on the average value of the height of the second image region in each shelf image region and the height of each shelf image region. The detection unit 14 detects the first image region in the image region to be processed by applying the first model to the image region to be processed determined by the determination unit 15. The second model is a model that identifies the second image region corresponding to the image of the group of items placed in the front in the target image. The first model is a model that identifies the first image region based on the main occupied space of the items in the target image.
[0037] The configuration of this detection device 10 makes it possible to improve the detection accuracy of the item group image area. That is, for example, if the average height of the second image area in the shelf image area is sufficiently small compared to the height of the shelf image area, there is a high probability that items located further back are captured in the shelf image area. When the second model is applied to such a shelf image area, there is a possibility that the image area corresponding to items located further back may not be detected accurately. On the other hand, when the first model is applied to such a shelf image area, there is a high probability that the image area corresponding to items located further back can be detected accurately. Conversely, if the average height of the second image area in the shelf image area is sufficiently large compared to the height of the shelf image area, there is a high probability that items located further back are hardly captured in the shelf image area. When the second model is applied to such a shelf image area, there is a high probability that the image area corresponding to the item group located at the front can be detected accurately. Therefore, since one model can compensate for the detection of targets that the other model is not good at detecting, the detection accuracy of the item group image area can be improved.
[0038] <Second Embodiment> The second embodiment relates to an embodiment that further elaborates on the content of the first embodiment.
[0039] Figure 3 is a block diagram showing an example of a detection device in the second embodiment. In Figure 3, the detection device 20 includes a detection unit (first detection unit) 21, a identification unit 22, an average value calculation unit 23, a detection unit (second detection unit) 24, and a determination unit 25. The detection device 20 acquires a captured image, similar to the detection device 10 in the first embodiment. This captured image is, for example, an image of a shelving unit. The following explanation assumes that the captured image is an image of a shelving unit. This shelving unit has multiple shelves.
[0040] Similar to the detection unit 11 in the first embodiment, the detection unit 21 detects a second image region in the captured image (i.e., the image of the shelves) by applying the second model to the captured image.
[0041] Figure 4A shows an example of an image of a shelf. Figure 4B shows an example of a second image region.
[0042] Figure 4A shows an image of a shelf displaying bottled beverages. The shelf in Figure 4A has four shelves. On the top three shelves, the bottles are arranged upright. On the bottom shelf, the bottles are arranged horizontally and stacked. The space above the bottles is narrow on the top three shelves, while the space above the bottles is wide on the bottom shelf. As a result, in the images of the top three shelves, most of the bottles at the back are hidden by the bottles at the front, while in the image of the bottom shelf, the bottles at the back are also visible.
[0043] Figure 4B shows the result of applying the second model to the shelf image in Figure 4A. In Figure 4B, the shaded area corresponds to the second image area.
[0044] As can be seen in Figure 4B, the second model accurately detects the image region corresponding to the group of PET bottles in the images of the top three shelves. As a result, the second model can accurately detect even image regions corresponding to narrow empty spaces, such as those sandwiched between two image regions of items.
[0045] On the other hand, as can be seen in Figure 4B, in the image of the bottom shelf, the image region of the item group corresponding to the group of PET bottles located at the front can be detected, but the image region of the item group corresponding to the group of PET bottles located at the back cannot be detected.
[0046] Returning to the explanation of Figure 3, the identification unit 22, similar to the identification unit 12 in the first embodiment, identifies multiple "shelf image regions" in the captured image that correspond to multiple shelf levels.
[0047] For example, as can be seen in Figure 4B, the lower line defining the second image region in the image of each shelf appears as a straight line approximately parallel to the shelf board. In other words, by identifying this straight line, it is possible to identify the line corresponding to the surface of the PET bottle in contact with the shelf board.
[0048] Therefore, the identification unit 22 may identify the lower line that defines the second image region in the image of each shelf, and identify the image region sandwiched between two adjacent lines as the "shelf image region".
[0049] Alternatively, for example, the identification unit 22 may directly identify the front image of the shelf board by pattern matching or the like. This front image of the shelf board can also be identified as lines corresponding to the surface of the PET bottle in contact with the shelf board. The identification unit 22 may then identify the image region enclosed by two adjacent lines as the "shelf tier image region". In Figure 4B, the image regions SA11, SA12, SA13, and SA14 enclosed by frames are examples of shelf tier image regions.
[0050] The average value calculation unit 23 calculates the average value of the height of the second image region in each shelf image region, similar to the average value calculation unit 13 in the first embodiment. The height of the second image region is the length of the second image region corresponding to the height direction of the item shelf.
[0051] Similar to the determination unit 15 in the first embodiment, the determination unit 25 determines the image area to be processed by the detection unit 24 from among a plurality of shelf image areas based on the average value of the height of the second image area in each shelf image area and the height of each shelf image area.
[0052] For example, as shown in Figure 3, the determination unit 25 has a specific processing unit 25A and a determination processing unit 25B.
[0053] The identification processing unit 25A identifies the length of each shelf image region corresponding to the height direction of the goods shelf as the height (α) of each shelf image region.
[0054] The decision processing unit 25B calculates a "reference value" for each identified shelf image region by multiplying the height (α) of each shelf image region by a predetermined ratio (for example, 0.6). Then, the decision processing unit 25B determines the image region to be processed by comparing the calculated "reference value" for each shelf image region with the average value of the height of the second image region corresponding to each shelf image region. For example, the decision processing unit 25B determines the shelf image region corresponding to the second image region with an average value smaller than the "reference value" as the image region to be processed by the detection unit 24. In the example in Figure 4B, the decision processing unit 25B determines the shelf image region SA14 as the image region to be processed by the detection unit 24.
[0055] The detection unit 24, similar to the detection unit 14 in the first embodiment, detects the first image region in the image region to be processed by applying the first model to the image region to be processed determined by the determination unit 25.
[0056] Figure 4C shows an example of the first image region. In Figure 4C, the shaded area in shelf image region SA14 corresponds to the first image region. As can be seen in Figure 4C, Model 1 accurately detects not only the item group image region corresponding to the group of PET bottles located at the front, but also the item group image region corresponding to the group of PET bottles located at the back, in shelf image region SA14 (i.e., the image of the bottom shelf). For reference, Figure 4C also shows the results of applying Model 1 to shelf image regions SA11, SA12, and SA13. As can be seen in Figures 4B and 4C, Model 1 may mistakenly detect image regions corresponding to narrow empty spaces SP1, such as those sandwiched between two item group image regions, as part of the item group image region.
[0057] <Third Embodiment> The third embodiment relates to the identification of vacant spaces.
[0058] Figure 5 is a block diagram showing an example of a detection device in the third embodiment. In Figure 5, the detection device 30 includes a detection unit (first detection unit) 11, a identification unit 12, an average value calculation unit 13, a detection unit (second detection unit) 14, a determination unit 15, an integration unit 31, and a space identification unit 32.
[0059] The integration unit 31 integrates the first image region in the processing target image region detected by the detection unit 14 and the second image region in each shelf image region other than the processing target image region detected by the detection unit 11 to obtain an "integrated image". For example, in the case of Figures 4B and 4C above, the second image regions in the shelf image regions SA11, SA12, and SA13 detected by the second model and the first image region in the shelf image region SA14 detected by the first model are integrated to form an "integrated image". Figure 6 shows an example of an integrated image.
[0060] The space identification unit 32 identifies empty spaces on each shelf based on the integrated image. For example, the space identification unit 32 may identify empty spaces by subtracting the integrated image from the shelf image area. In Figure 6, for example, the areas enclosed by rectangular frames (spaces SP1, SP2, SP3, SP4) correspond to empty spaces.
[0061] As described above, the detection device 30 in the third embodiment identifies empty spaces based on an integrated image obtained by integrating the image regions obtained by applying the first model and the second model, which are the shelf image regions for which each model excels, so that empty spaces can be identified with high accuracy.
[0062] Although this description assumes that the integration unit 31 and the space identification unit 32 are applied to the detection device 10 of the first embodiment, this disclosure is not limited thereto. The integration unit 31 and the space identification unit 32 may also be applied to the detection device 20 of the second embodiment.
[0063] <Other Embodiments> Figure 7 shows an example of the hardware configuration of a detection device. In Figure 7, the detection device 100 has a processor 101 and a memory 102. The processor 101 may be, for example, a microprocessor, an MPU (Micro Processing Unit), or a CPU (Central Processing Unit). The processor 101 may include multiple processors. The memory 102 is composed of a combination of volatile memory and non-volatile memory. The memory 102 may include storage located away from the processor 101. In this case, the processor 101 may access the memory 102 via an I / O interface that is not shown.
[0064] The detection devices 10, 20, and 30 of the first to third embodiments may each have the hardware configuration shown in Figure 7. The detection units 11 and 21, the identification units 12 and 22, the average value calculation units 13 and 23, the detection units 14 and 24, the determination units 15 and 25, the integration unit 31, and the space identification unit 32 of the detection devices 10, 20, and 30 of the first to third embodiments may be realized by the processor 101 reading and executing a program stored in the memory 102. The program can be stored using various types of non-transitory computer-readable media and supplied to the detection devices 10, 20, and 30. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives) and magneto-optical recording media (e.g., magneto-optical disks). Furthermore, examples of non-transitory computer-readable media include CD-ROMs (Read Only Memory), CD-Rs, and CD-R / Ws. Furthermore, examples of non-transitory computer-readable media include semiconductor memory. Semiconductor memory includes, for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, and RAM (Random Access Memory). Programs may also be supplied to detection devices 10, 20, and 30 by various types of transient computer-readable media. Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can supply programs to detection devices 10, 20, and 30 via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.
[0065] Although the present invention has been described above with reference to embodiments, the present invention is not limited thereto. Various modifications to the structure and details of the present invention can be made that are understandable to those skilled in the art within the scope of the invention. [Explanation of symbols]
[0066] 10 Detection device 11. Detection Unit (First Detection Unit) 12 Specific part 13. Average value calculation section 14. Detection Unit (Second Detection Unit) 15. Decision Section 20 Detection device 21 Detection Unit (First Detection Unit) 22 Specific section 23. Average value calculation section 24 Detection unit (second detection unit) 25 Decision Section 25A Specific Processing Unit 25B Decision Processing Unit 30 Detection device 31. Integration Department 32 Space Identification Section
Claims
1. A first detection unit detects the second image region in a captured image of a shelf using the second model, which is one of a first model that identifies a first image region based on the main occupied space of an item group in a target image, and a second model that identifies a second image region corresponding to an image of an item group arranged in the front of the target image. A selection unit that identifies multiple shelf image regions corresponding to multiple shelf levels of the item shelf in the captured image, An average value calculation unit that calculates the average value of the height of the second image region corresponding to the height direction of the item shelf in each shelf image region, A determination unit that determines the image region to be processed from among the plurality of shelf image regions based on the average value in each shelf image region and the height of each shelf image region, A second detection unit detects the first image region in the image region to be processed by applying the first model to the image region to be processed, A detection device equipped with the following.
2. The determination unit determines, among the plurality of shelf image regions, the shelf image region corresponding to the second image region having the average value less than a predetermined percentage of the height of the shelf image region, as the image region to be processed. The detection device according to claim 1.
3. The aforementioned determination unit, A determination processing unit that determines the length of each shelf image region corresponding to the height direction of the item shelf as the height of each shelf image region, A determination processing unit determines the image region to be processed by comparing a value obtained by multiplying the height of each identified shelf image region by a predetermined percentage with the average value of the height of the second image region corresponding to each shelf image region. Equipped with, The detection device according to claim 2.
4. The identifying unit identifies a plurality of lines in the second image region that correspond to the surface of an article in contact with the shelf board of the article rack, and identifies the plurality of shelf image regions by dividing the captured image by the plurality of lines. The detection device according to claim 1.
5. The detection device according to claim 1, further comprising an integration unit that integrates the first image region in the processing target image region and the second image region in each shelf image region other than the processing target image region to obtain an integrated image.
6. The system further includes a space identification unit that identifies empty spaces on each shelf where no items are placed, based on the aforementioned integrated image. The detection device according to claim 5.
7. A first model identifies a first image region based on the main occupied space of an item group in the target image, and a second model identifies a second image region corresponding to an image of an item group placed in the foreground of the target image. Using the second model, the second image region in a captured image of an item shelf is detected. In the aforementioned captured image, a plurality of shelf image regions corresponding to each of the plurality of shelf levels of the item shelf are identified, To calculate the average value of the height of the second image region corresponding to the height direction of the item shelf in each shelf image region, Based on the average value in each shelf image region and the height of each shelf image region, the image region to be processed is determined from among the plurality of shelf image regions. The first image region in the image region to be processed is detected by applying the first model to the image region to be processed, A detection method that includes [this].
8. The above determination includes determining, among the plurality of shelf image regions, the shelf image region corresponding to the second image region having the average value less than a predetermined percentage of the height of the shelf image region, as the image region to be processed. The detection method according to claim 7.
9. A first model identifies a first image region based on the main occupied space of an item group in the target image, and a second model identifies a second image region corresponding to an image of an item group placed in the foreground of the target image. Using the second model, the second image region in a captured image of an item shelf is detected. In the aforementioned captured image, a plurality of shelf image regions corresponding to each of the plurality of shelf levels of the item shelf are identified, To calculate the average value of the height of the second image region corresponding to the height direction of the item shelf in each shelf image region, Based on the average value in each shelf image region and the height of each shelf image region, the image region to be processed is determined from among the plurality of shelf image regions. The first image region in the image region to be processed is detected by applying the first model to the image region to be processed, A program that causes a detection device to perform a process that includes [a specific action].
10. The above determination includes determining, among the plurality of shelf image regions, the shelf image region corresponding to the second image region having the average value less than a predetermined percentage of the height of the shelf image region, as the image region to be processed. The program according to claim 9.