Image processing device

The image processing apparatus uses AI to detect and synthesize images of similar packages by determining commonalities in package type and position, addressing inaccuracies in existing synthesis methods and enhancing loading rate calculations.

JP7708017B2Active Publication Date: 2025-07-15TOYOTA JIDOSHA KK
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2022106560
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2022-06-30
Publication Date
2025-07-15
Estimated Expiration
2042-06-30

AI Technical Summary

Technical Problem

Existing image synthesis technologies struggle to accurately combine images when multiple packages of similar types and dimensions are present, leading to inaccurate loading rate calculations due to undetected matching points.

Method used

An image processing apparatus that utilizes an AI model to detect package types and coordinates, determines common packages based on type and position, and synthesizes images by overlapping common packages, correcting for differences in imaging distance and angle.

Benefits of technology

Accurately synthesizes images even with similar packages, improving loading rate calculations by enhancing detection accuracy and simplifying user operation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007708017000001
    Figure 0007708017000001
  • Figure 0007708017000002
    Figure 0007708017000002
  • Figure 0007708017000003
    Figure 0007708017000003
Patent Text Reader

Abstract

To provide an image processing device which can accurately synthesize images even when luggage items of the same type or the similar size exist in a plurality of images obtained by capturing images of the plurality of luggage items.SOLUTION: An image processing device synthesizes a plurality of images obtained by capturing images of a plurality of luggage items. The image processing device comprises: an image acquisition unit that acquires the plurality of images; a luggage detection unit that detects the type and position coordinates of each of the plurality of luggage items from the plurality of images; a luggage dimension calculation unit that calculates the dimension of the luggage from its position coordinates; a luggage determination unit that determines on the basis of the detected type and dimension of the luggage whether or not a common luggage is included between the plurality of images; and an image synthesis unit that superimposes the common portions of the luggage item on top of the other so as to synthesize the plurality of images when it is determined by the luggage determination unit that the common luggage is included between the plurality of images.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an image processing apparatus for synthesizing a plurality of images.

Background Art

[0002] In order to improve logistics efficiency, it is required to automatically calculate the loading rate of the goods loaded on the loading platform of a truck or the like. Although the loading rate can be predicted from a database, errors occur due to gaps at the time of loading. In addition, since there are a large number of goods, it is difficult to manage them by RFID. Therefore, although it is desirable to automatically calculate the loading rate from the captured image, when the distance between the imager and the loading platform is short, the entire loading platform cannot be imaged, so it is necessary to combine and synthesize the images captured separately.

[0003] Regarding the above image synthesis technology, Patent Document 1 discloses a technology for detecting the overlap position (corresponding points) between the images to be synthesized, and finding the point that most matches the other image for the points where the edge of one image is strong enough at a predetermined value or more, and synthesizing them.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When trying to use the technology of Patent Document 1, for example, to calculate the loading rate of the goods loaded on the loading platform of a truck, when a plurality of goods of the same type and similar dimensions are loaded on the loading platform, there are a plurality of similar points in the images, and the matching points between the images cannot be accurately detected, and the images cannot be synthesized.

[0006] The present invention has been made to solve the above problems, and an object thereof is to provide an image processing apparatus capable of accurately synthesizing images even when there are a plurality of packages of the same type and similar dimensions among a plurality of images obtained by imaging a plurality of packages.

Means for Solving the Problems

[0007] In order to solve the above problems, the image processing apparatus of the present invention has the following configuration. The effects exhibited by each configuration will also be described together.

[0008] The image processing apparatus according to the present invention is an image processing apparatus that synthesizes a plurality of images obtained by imaging a plurality of packages, and includes an image acquisition unit that acquires a plurality of images, a package detection unit that detects the type and position coordinates of each of the plurality of packages from the plurality of images, a package dimension calculation unit that calculates the dimensions of the packages from the position coordinates of the packages, a package determination unit that determines whether or not the plurality of images include packages common to each other based on the detected types and dimensions of the packages, and an image synthesis unit that synthesizes the plurality of images by overlapping the portions of the common packages when the package determination unit determines that the plurality of images include packages common to each other.

[0009] According to the present invention, instead of using the detection of the overlap position between images as in the prior art, it is determined whether there are common packages between images by recognizing the degree of overlap of the images based on the types and position coordinates of the packages. Therefore, even when there are a plurality of packages of the same type and similar dimensions among a plurality of images, it is possible to accurately synthesize the images.

[0010] As a more preferable image processing apparatus, when the package determination unit determines that the plurality of images include packages common to each other, the package determination unit further determines whether or not other packages in contact with the common packages are common among the plurality of images.

[0011] According to this configuration, it becomes possible to confirm that there are a plurality of packages common among the images, and it becomes possible to more accurately calculate the commonality among the images.

[0012] As a more preferable image processing apparatus, the package detection unit detects the type and position coordinates for each package column in which a plurality of packages are stacked in the vertical direction, and the package determination unit determines, for each package column, whether a package is common among a plurality of images. When it is determined that a package common among a plurality of images is included, it is further determined whether another package in contact with either the upper or lower side of the common package is common among the plurality of images.

[0013] Packages loaded on the loading platform of a motor vehicle such as a truck are often placed along the width direction of the loading platform with a plurality of package columns stacked in the vertical direction. By adopting the above configuration, package determination is performed according to a certain rule, so it becomes possible to provide simplicity to the user of the image processing apparatus.

[0014] As a more preferable image processing apparatus, the package determination unit determines, among a plurality of images, packages having the same type and a dimensional difference within a predetermined threshold as common packages.

[0015] According to this configuration, even when the distance and angle between the imaging device and the imaging target are somewhat different among the plurality of images when obtaining the plurality of images, the difference can be corrected and accurate image synthesis can be performed.

[0016] As a more preferable image processing apparatus, the package detection unit is one that has learned, as teacher data, an image obtained by imaging a plurality of packages, a method for extracting the packages included in the image, and a method for specifying the type of the packages. The package detection unit detects the type and position coordinates of each of the plurality of packages from the plurality of images acquired by the image acquisition unit.

[0017] According to this configuration, it becomes possible to detect packages by utilizing an AI model that has been learned in advance using, for example, a neural network for a large number of packages, and a significant improvement in detection accuracy can be expected.

[0018] As a more preferable image processing apparatus, the plurality of packages are a plurality of packages loaded on the loading platform of a motor vehicle, and the plurality of images are a plurality of images obtained by dividing the loading platform in the longitudinal direction of the motor vehicle.

[0019] On the loading platform of a truck responsible for logistics, many packages of the same type and similar dimensions are often loaded. Therefore, the present invention can be suitably adopted for an object having such a configuration.

Advantages of the Invention

[0020] According to the present invention, even when there are a plurality of packages of the same type and similar dimensions among a plurality of images obtained by imaging a plurality of packages, it becomes possible to accurately perform image synthesis.

Brief Description of the Drawings

[0021]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0022] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this embodiment, the cargo bed portion of a truck (automobile) loaded with a plurality of packages is taken as the imaging target, and an example will be described in which a plurality of images obtained as a result of imaging are images obtained by dividing the cargo bed of the truck in the front-rear direction of the truck.

[0023] FIG. 1 is a block diagram showing the hardware configuration of an image processing apparatus 1 according to an embodiment of the present invention. The image processing apparatus 1 includes a ROM (Read Only Memory) 11 which is a storage device capable of reading data, a RAM (Random Access Memory) 12 capable of both writing and reading data, a CPU (Central Processing Unit) 13 which performs arithmetic operations on parameters necessary for vehicle control based on data, and a communication module 14 which is an interface for transmitting and receiving data.

[0024] The image processing apparatus 1 is connected to an imaging apparatus 3 and an external server 4 via a communication path 2. The communication path 2 can be composed of a wireless network such as Wi-Fi (registered trademark). The image processing apparatus 1 receives a plurality of images obtained by imaging an imaging target from the imaging apparatus 3. Also, between the image processing apparatus 1 and the external server 4, various types of information including calculation results and teacher data for machine learning described later are transmitted and received. The external server 4 may be installed in a control center as a computer equipped with a storage, or may be implemented as a virtual machine on the cloud.

[0025] The image processing apparatus 1 receives data via the communication path 2. Then, in response to the reception, the received data is expanded in the RAM 12. Then, based on the data written in the RAM 12 and the data read from the ROM 11, the CPU 13 performs arithmetic operations. Also, a program is pre-installed in the image processing apparatus 1, and the CPU 13 can execute the processes described below by executing the program. Hereinafter, while appropriately omitting well-known techniques in the field of computers such as these, the details of each part will be described.

[0026] Next, the functional configuration of the image processing apparatus 1 will be described using the functional block diagram of FIG. 2. As shown in FIG. 2, the image processing apparatus 1 includes an image acquisition unit 15, a baggage detection unit 16, a baggage size calculation unit 17, a baggage determination unit 18, an image synthesis unit 19, and an output unit 20. Further, a baggage information database (DB) 21 is stored in the storage area of the image processing apparatus 1.

[0027] The functional block diagram shown in FIG. 2 is an example, and the units and names of functions are not limited thereto. For example, the image acquisition unit 15 and the output unit 20 may be collectively treated as a data transmission / reception unit.

[0028] The image acquisition unit 15 acquires a plurality of images captured by the imaging device 3 via the communication path 2. The baggage detection unit 16 analyzes the plurality of images acquired by the image acquisition unit 15 for each image, and detects the type and position coordinates of the baggage included in the image. The baggage size calculation unit 17 calculates the size of each baggage from the position coordinates of each baggage detected by the baggage detection unit 16. The baggage determination unit 18 determines whether or not there is a common baggage among the plurality of images based on the type of each baggage detected by the baggage detection unit 16 and the size of each baggage calculated by the baggage size calculation unit 17. The image synthesis unit 19 synthesizes a plurality of images determined to include a common baggage by the baggage determination unit 18. The output unit 20 outputs the synthesized image data, the baggage information included in the image, the results calculated by each functional unit, etc. to the external server 4.

[0029] In the luggage information DB21, various data regarding the luggage, which are obtained by performing machine learning in advance on the dimensions of the luggage included in the image and the detection results of the types of the luggage, with the images of a plurality of luggage captured as teacher data, are stored. Also, in the present embodiment, as will be described later, the luggage detection unit 16 constitutes an AI model (learning model). The AI model is a model that has performed machine learning in advance on the method of extracting the luggage included in the image (recognition of the luggage and detection process of the coordinates on the image of the luggage) and the process of specifying the type of the extracted luggage, using the image data of the luggage that can be the imaging target as teacher data. Specifically, the feature amounts of the luggage are machine-learned using the teacher data so that the feature amounts such as the dimensions or shapes of each luggage are extracted. As a result, it has become possible to detect the luggage included in the image with high precision.

[0030] Next, the process of detecting the type and position coordinates for each luggage in the plurality of images obtained by the image processing apparatus will be described with reference to FIG. 3. Note that in the present embodiment, the process of generating one image by synthesizing two images including a common luggage between the two images will be described, but it goes without saying that the number of images to be synthesized and the number of generated images are not limited to the present embodiment.

[0031] The image acquisition unit 15 acquires the first image shown in FIG. 3(a) and the second image shown in FIG. 3(b) from the imaging apparatus 3. As described above, the images acquired in the present embodiment are part of the loading platform of a truck on which a plurality of luggage are loaded. The first image shown in FIG. 3(a) is an image obtained by imaging a state in which a luggage row (L1, L2, L3 ···) in which a plurality of luggage 100a are stacked in the vertical direction is loaded along the rear from the front of the loading platform. The second image shown in FIG. 3(b) is a luggage row (L k , L k+1 , L k+2···) is an image obtained by imaging the state in which it is loaded from the middle of the loading platform along the rear. Therefore, the second image shown in FIG. 3(b) images a part of the loading platform further to the rear than the first image shown in FIG. 3(a). Thus, when the first image and the second image are combined, the part including the right end of the first image and the part including the left end of the second image are overlapped and combined as a common part.

[0032] Here, the package 100a in the present embodiment is defined as being composed of a skid portion 102a for inserting the forks of a forklift and carrying it, and a load portion 101a placed on the skid portion 102a. However, it goes without saying that the objects that can be defined as packages are not limited to those that can be loaded on the loading platform of a truck, and there are infinitely many objects that can be defined as packages. Also, the load portion 101a and the skid portion 102a may be distinguished and each defined as a package. Thus, the definition of a package as the scope of application of the present invention can be arbitrarily determined.

[0033] The image acquisition unit 15 transmits the acquired image data to the package detection unit 16. The package detection unit 16 constitutes an AI model for package detection as described above. Using the image data received from the image acquisition unit 15 as input, it outputs the number, type, position coordinates, etc. of the packages included in the image. Specifically, using the learned AI model, it extracts the features of the objects included in the image in pixel units, and automatically processes color, brightness, distortion, noise, distinction of ranges, etc. to visualize and classify what is shown in the "image classification process", automatically recognizes the subject and detects the existence range of the objects in the image in the "image detection process", and for each image, it executes the "image segmentation process" that clearly demarcates the boundary lines of the subject to identify what is in the image area.

[0034] For example, taking the skid part 102a in Fig. 3(a) as an example, for the luggage detection unit 16, features such as the "dimension ratio in the vertical and horizontal directions", "position of the fork insertion hole and area ratio to the whole", "brightness, color, surface unevenness, transparency, etc. of the constituent material" of the skid part 102a are used as the teacher data of the luggage image in the image, and an AI model (learning model) that has been pre-trained for the extraction process of the luggage included in the image is constructed. When an image to be detected for luggage is input into the constructed AI model, the AI model extracts the feature quantities of the luggage from the image, and based on the extracted feature quantities, it can detect that the object marked with code 102a is the skid part. Thereby, the luggage detection unit 16 (AI model) can detect the position coordinates of the luggage 100a in the image and identify the type of the luggage 100a by detecting the skid part included in the image. The same applies to the load part 101a placed on the skid part 102a. Naturally, for other objects as well, by giving teacher data in advance and training, they can be detected with high accuracy.

[0035] The luggage detection unit 16 detects the type of each luggage as a result of the above image analysis. Then, as shown in Fig. 3(b), for each luggage, luggage segmentation distinguished for each luggage is assigned (luggage segmentation 200a for luggage 100a, luggage segmentation 200b for luggage 100b). Also, during the above series of processes, the image is pixel-analyzed, and the position coordinates for each luggage segmentation are also calculated. The origin of the position coordinates is, for example, the uppermost left point of each image.

[0036] Through the above processing, the luggage detection unit 16 detects the type and position coordinates of each luggage, and transmits the detection results to the luggage dimension calculation unit 17. The luggage dimension calculation unit 17 executes the following processing based on the received detection results.

[0037] Figure 4(a) is a list of the luggage information detected by the luggage detection unit 16, including the luggage id, the type (label) of the luggage, the detection accuracy, and the coordinates of the four sides (left, top, right, bottom) of the detected luggage. Figure 4(a) shows that the data is listed along the id assigned when the luggage detection unit 16 recognizes each piece of luggage, for example, and it is difficult to handle the data as it is. Therefore, the luggage dimension calculation unit 17 first organizes the data in Figure 4(a) to make it easier to handle.

[0038] Specifically, for example, the coordinates of the left side of each piece of luggage are sorted. Then, as shown in Figure 4(b), from the chart, the luggage with the same left side coordinates will be displayed. From this example, it can be seen that the luggage assigned with id: 21, 19, 3, and 1 is the leftmost luggage row in the captured image. The skid with id: 1 is directly placed on the bottom surface of the loading platform, the pori with id: 19 is placed on it, the skid with id: 3 is placed on it, and the pori with id: 21 is placed on it (note that the coordinate origin is the point at the upper left of the image).

[0039] In this way, the luggage dimension calculation unit 17 sorts the luggage information and organizes it so that it can be handled as data for each luggage row. Then, for each luggage row, the dimensions (height and width) of each piece of luggage are calculated by obtaining the difference in the position coordinates. The luggage dimension information calculated in this way is further added to the luggage information in Figure 4(b) and sent to the luggage determination unit 18.

[0040] Referring to a series of data in the table of Fig. 4(b) where the left coordinate is "911", among the six detected packages, the right coordinates of five packages are "1024", and the remaining one is also "1021". Also, the package widths obtained from this coordinate difference are "113" and "110", which are extremely small compared to other packages. Thus, when the calculated package width is extremely small compared to the widths of other packages, the package row can be excluded from the determination target by the package determination unit 18 as if the package row is cut off in the image. Regarding the exclusion criterion in this case, it may be preset, for example, to 50% or less of the average value of the detected package widths, or it may be changed in real time according to the situation.

[0041] Next, the package determination unit 18 that has received package information from the package size calculation unit 17 determines whether there are packages common to multiple images, and when the package determination unit 18 determines that there are packages common to multiple images, the image synthesis unit 19 performs image synthesis. This will be described with reference to Fig. 5.

[0042] First, the package determination unit 18 recognizes from the package information received from the package size calculation unit 17 that the first image shown in Fig. 5(a) is an image obtained by imaging the left part of the loading platform relative to the second image shown in Fig. 5(b), and arranges the first image on the left and the second image on the right. Also, as a result of the processing by the package size calculation unit 17, it is set that the package rows to be determined in the first image are L1, L2, and L3.

[0043] Here, the package determination unit 18 in this embodiment is set to recognize all the packages included in each package row as package row segmentations (recognize the packages included in the package row L3 as the package row segmentation 300a). Then, the package determination unit 18 executes a process of determining whether there are packages common to multiple images for each package row segmentation.

[0044] First, the package determination unit 18 extracts the types and dimensions of the packages 110a to 140a included in the rightmost package row segmentation 300a of the package rows to be determined in the first image from the package information. Then, it starts determining whether the package 110a placed at the uppermost stage of this package row segmentation 300a is included in the second image. Hereinafter, the type and dimensions of the package 110a are referred to as reference package information.

[0045] Based on the above reference package information, the types and dimensions of the package 100b placed at the uppermost stage of the leftmost package row segmentation of the package rows to be determined in the second image are detected. If either the type or the dimensions of the detected package do not match the reference package information, the types and dimensions of the package placed one stage below the determined package 100b are detected, and the same determination is made. When the determination is made up to the package placed at the lowermost stage of the package row segmentation, the process moves to the package placed at the uppermost stage of the package row segmentation one column to the right of that package row segmentation, and the determination continues. In addition, in the comparison of the dimensions of the packages between the images, each image may be calibrated to correct the deviation due to differences in the distance between the imaging device and the imaging target area and the optical axis angle when each image is captured. Also, a certain threshold value that can be determined to have the same dimensions may be set.

[0046] In this embodiment, as a result of repeating the above process, the package determination unit 18 determines that the type and dimensions of the package 110b at the uppermost stage of the second leftmost package row segmentation 300b of the package rows to be determined in the second image match the reference package information. That is, it is assumed that the leftmost package row in the second image is determined to be the same as the package row L2 in the first image.

[0047] Then, the package determination unit 18 refers to the first image again, and this time extracts the package information of the package 120a placed second from the top in the package row segmentation 300a, and determines whether it matches the types and dimensions of the package 120b placed second from the top in the package row segmentation 300b of the second image.

[0048] Through the above processing, the package determination unit 18 determines that, between the first image and the second image, two packages adjacent vertically (package 110a and 120a in the first image and package 110b and 120 in the second image) exist in a common package row segmentation (300a in the first image and 300b in the second image), and transmits the determination result to the image synthesis unit 19.

[0049] When the image synthesis unit 19 receives the determination result from the package determination unit 18, it overlaps the two images so that the package row segmentation 300a of the first image and the package row segmentation 300b of the second image overlap, and generates a synthesized image shown in FIG. 5(c).

[0050] In the above, when two packages adjacent vertically included in the package row segmentation match, it is determined that the package row segmentations match. However, the number of packages to be compared is not limited to this, and the above determination process may be similarly executed for package 130a and 140a in the first image and package 130b and 140b in the second image. The accuracy of image determination improves as the number of packages to be compared increases.

[0051] In the above description, for the sake of simplicity, it is assumed that the first image is a part including the head of the loading platform and the second image is behind the loading platform compared to the first image. However, the package determination unit 18 does not necessarily understand the positional relationship between the received image data. Therefore, there may be a case where the positional relationship is the reverse of the above-described first image and second image. Therefore, if, as a result of performing the determination process on all the packages included in a certain package row segmentation of the first image arranged on the left side, it is not determined that the second image includes matching packages, the possibility that the second image is an image capturing a part in front of the loading platform compared to the first image is also considered, and in the first image, the packages included in the package row segmentation one column to the left are newly set as determination targets, and the determination process is continued.

[0052] Finally, the output unit 20 outputs the determination result to the package information DB 21 and / or the external server 4, and updates the teacher data or the like.

[0053] The above-described series of determination processes will be described with reference to the flowchart of FIG. 6. The process starts when the image acquisition unit 15 acquires a plurality of images from the imaging device 3.

[0054] In step S601, the package detection unit 16 detects the type and position coordinates of the package included in the image for each of the plurality of images acquired from the image acquisition unit 15 using the above-described AI model.

[0055] In step S602, the package detection unit 16 calculates the average width of the packages for each type of detected package.

[0056] In step S603, the package dimension calculation unit 17 sorts the package information as described with reference to FIG. 4 and organizes it into information for each package row. At this time, the package row to be excluded from the determination target by the package determination unit 18 may be selected using the average width of each type of package calculated in step S602.

[0057] In step S604, the package determination unit 18 extracts the packages within the rightmost package row segmentation in the first image (the image that captures the frontmost part of the loading platform among the plurality of acquired images), and calculates the package information as described with reference to FIG. 5.

[0058] In steps S605 - S606, the package determination unit 18 detects the package information of the packages to be determined in the second image (the image that captures the rear part of the loading platform more than the first image), and determines whether there is a package common to the packages in the package row segmentation extracted in step S604 in the second image based on the package information received from the package dimension calculation unit 17.

[0059] In step S606, if it is determined that there is no common package between the first image and the second image, the package information within the package row segmentation one column to the left in the first image is calculated, and similarly, the determination as to whether there is a common package in the second image is continued.

[0060] In step S606, if it is determined that there is a common package between the first image and the second image, the process proceeds to step S607. In this step S607, the package determination unit 18 further determines whether the packages one row below the package determined to be common in step S606 are common for each of the first image and the second image.

[0061] In step S607, if it is determined that the packages one row below match between the images, in step S608, the image is synthesized by the image synthesis unit 19 that has received the result. If it is determined that the packages one row below do not match between the images, it is determined that the package determined to be common in step S606 happened to be present in both images, and the process by the package determination unit 18 proceeds to step S609. After that, it is as described above.

[0062] When the image is synthesized by the image synthesis unit 19, a series of processing results are output by the output unit 20 to the package information DB21 and / or the external server 4, and the process ends.

[0063] As described above, according to the present invention, since the determination as to whether there is a common package between the images is executed based on the dimensions based on the type and position coordinates of the package, it is possible to improve the accuracy of the determination even in a case where there are many similar packages. Moreover, when it is determined that there is a common package between the images, by determining whether there is also a commonality in the packages in contact with that package, it is possible to further improve the determination accuracy. Furthermore, when detecting the type and dimensions of the package, by using, as the package detection unit, an AI model generated by machine learning the type and dimensions of the package using, as teacher data, images obtained by imaging a plurality of packages in advance, it is possible to extremely improve the accuracy of package detection.

[0064] As described above in detail with respect to the embodiments of the present invention, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention described in the claims. The present invention can add the configuration of one embodiment to the configuration of another embodiment, replace the configuration of one embodiment with that of another embodiment, or delete a part of the configuration of one embodiment.

Description of Reference Numerals

[0065] 1 Image processing apparatus, 15 Image acquisition unit, 16 Luggage detection unit, 17 Luggage dimension calculation unit, 18 Luggage determination unit, 19 Image synthesis unit

Claims

1. An image processing apparatus for synthesizing a plurality of images obtained by imaging a plurality of packages, comprising: an image acquisition unit that acquires the plurality of images; a package detection unit that detects the type and position coordinates of each of the plurality of packages from the plurality of images; a package size calculation unit that calculates the size of the package from the position coordinates of the package; a package determination unit that determines whether a package common to the plurality of images is included based on the detected type and size of the package; an image synthesis unit that synthesizes the plurality of images by overlapping portions of the common package when the package determination unit determines that a package common to the plurality of images is included; and when the package determination unit determines that a package common to the plurality of images is included, the package determination unit further determines whether another package in contact with the common package is common among the plurality of images. An image processing apparatus characterized by the above.

2. The image processing apparatus according to claim 1, wherein the package detection unit detects the type and position coordinates for each package row in which a plurality of packages are stacked in the vertical direction, and the package determination unit determines whether a package is common among the plurality of images for each package row, and when it is determined that a package common among the plurality of images is included, further determines whether another package in contact with either above or below the common package is common among the plurality of images. An image processing apparatus characterized by the above.

3. The image processing apparatus according to claim 1, wherein the package determination unit determines, among the plurality of images, a package having a matching type and a difference in the size within a predetermined threshold as a common package. An image processing apparatus characterized by the above.

4. The image processing apparatus according to claim 1, wherein the package detection unit has learned, using an image of the plurality of packages taken as teacher data, a method for extracting the packages included in the image and a method for specifying the types of the packages, and the package detection unit detects the type and position coordinates of each of the plurality of packages from the plurality of images acquired by the image acquisition unit. An image processing apparatus characterized by the above.

5. The image processing apparatus according to claim 1, wherein the plurality of packages are a plurality of packages loaded on a loading platform of an automobile, and the plurality of images are a plurality of images obtained by dividing the loading platform in the front - rear direction of the automobile. An image processing apparatus characterized by the above.

Citation Information

Patent Citations

  • Panorama image synthesis system and panorama image synthesis method

    JP1997091407A

  • Image processing apparatus

    JP2017014014A

  • Image recognition system

    JP2020161196A

  • Loading space recognition device, loading space recognition method, and loading space recognition program

    JP2021189666A

  • Information processing device, information processing method, and program

    JP2022033149A