Image analysis device, image analysis method, and image analysis program

Abstract

The present invention reduces the data volume of video data to transmit in a case where an object to be inspected is imaged with video and the captured video data is analyzed by a transmission-receiving image analysis device. This image analysis device comprises: an identification unit that, if first image data that has been encoded at a first compression rate has been decoded to acquire first decoded data, identifies regions for specifying an object region including an object to be inspected and a reference region including a reference line; a specifying unit for using the identified regions as well as information indicating a configuration of a structure to which the object to be inspected is attached to specify the object region and the reference region; and a measurement unit for measuring the location of the object to be inspected based on the reference line, if a compression rate map is generated in which the object region and the reference region have been changed to a second compression rate smaller than the first compression rate, and if second decoded data obtained by decoding second image data that has been encoded using the compression rate map is acquired.

Description

[Technical Field]

[0001] The present invention relates to an image analysis device, an image analysis method, and an image analysis program. [Background technology]

[0002] There is a known analysis technique for inspecting an object by capturing video of the object and analyzing the video data in an image analysis device at the destination. This analysis technique allows for pixel-level inspection, for example, to check whether the object is positioned according to design data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-112440 [Patent Document 2] Patent Publication No. 2021-057769 Summary of the Invention [Problem to be solved by the invention]

[0004] On the other hand, with the above analysis technology, it is necessary to increase the pixel resolution during shooting in order to measure the position of the object being inspected with high accuracy. However, in this case, the amount of video data to be transmitted increases, which puts a strain on the network bandwidth.

[0005] In one aspect, an object of the present invention is to reduce the amount of moving image data to be transmitted when an object to be inspected is photographed as a moving image and the moving image data is analyzed by an image analysis device at the destination. [Means for solving the problem]

[0006] According to one aspect, an image analysis device includes: an identification unit that decodes the first image data encoded at the first compression rate, and when acquiring first decoded data, analyzes the first decoded data to identify an object region including an object to be inspected and a reference region including a reference line; an identification unit that identifies the object region and the reference region using the identified region and information indicating the configuration of a structure to which the object to be inspected is attached; a measurement unit that generates a compression rate map in which the object area and the reference area are changed to a second compression rate that is smaller than the first compression rate, and when second decoded data is obtained by decoding second image data encoded using the compression rate map, measures the position of the object to be inspected based on the reference line by analyzing the second decoded data. [Effects of the Invention]

[0007] When an object to be inspected is photographed as a moving image and the moving image is analyzed by an image analyzing device at the destination, the amount of moving image data to be transmitted can be reduced. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration of an image processing system. [Figure 2] FIG. 2 is a diagram illustrating an example of the hardware configuration of the encoding device and the image analysis device. [Figure 3] FIG. 3 is a first diagram illustrating an example of the functional configuration of the encoding device and the image analysis device of the image processing system according to the first embodiment. [Figure 4] FIG. 4 is a first diagram showing a specific example of image processing by the image processing system according to the first embodiment. [Figure 5] FIG. 5 is a second diagram showing a specific example of image processing by the image processing system according to the first embodiment. [Figure 6] FIG. 6 is a third diagram showing a specific example of image processing by the image processing system according to the first embodiment. [Figure 7]FIG. 7 is a fourth diagram showing a specific example of image processing by the image processing system according to the first embodiment. [Figure 8] FIG. 8 is a first flowchart showing the flow of image processing by the image processing system according to the first embodiment. [Figure 9] FIG. 9 is a second flowchart showing the flow of image processing by the image processing system according to the first embodiment. [Figure 10] FIG. 10 is a first diagram showing an example of a change in compression ratio. [Figure 11] FIG. 11 is a first diagram showing a specific example of image processing by the image processing system according to the second embodiment. [Figure 12] FIG. 12 is a second diagram showing a specific example of image processing by the image processing system according to the second embodiment. [Figure 13] FIG. 13 is a third diagram showing a specific example of image processing by the image processing system according to the second embodiment. [Figure 14] FIG. 14 is a fourth diagram showing a specific example of image processing by the image processing system according to the second embodiment. [Figure 15] FIG. 15 is a first flowchart showing the flow of image processing by the image processing system according to the second embodiment. [Figure 16] FIG. 16 is a second flowchart showing the flow of image processing by the image processing system according to the second embodiment. [Figure 17] FIG. 17 is a diagram illustrating an example of the functional configuration of an encoding device and an image analysis device in an image processing system according to the third embodiment. [Figure 18] FIG. 18 is a first diagram showing a specific example of image processing by the image processing system according to the third embodiment. [Figure 19] FIG. 19 is a second diagram showing a specific example of image processing by the image processing system according to the third embodiment. [Figure 20] FIG. 20 is a third diagram showing a specific example of image processing by the image processing system according to the third embodiment. [Figure 21]FIG. 21 is a fourth diagram showing a specific example of image processing by the image processing system according to the third embodiment. [Figure 22] FIG. 22 is a diagram illustrating an example of the functional configuration of an encoding device and an image analysis device in an image processing system according to the fourth embodiment. [Figure 23] FIG. 23 is a second diagram showing an example of a change in compression ratio. [Figure 24] FIG. 24 is a third diagram showing an example of a change in compression ratio. [Figure 25] FIG. 25 is a fourth diagram showing an example of changes in compression ratio. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, each embodiment will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0010] [First embodiment] <Image processing system configuration> First, the system configuration of the image processing system will be described. Fig. 1 is a diagram showing an example of the system configuration of the image processing system. As shown in Fig. 1, the image processing system 100 has an imaging device 110, an encoding device 120, and an image analysis device 130. In the image processing system 100, the encoding device 120 and the image analysis device 130 are connected to each other via a network 180 so as to be able to communicate with each other.

[0011] Imaging device 110 captures images at a predetermined frame rate and transmits the video data to encoding device 120. Multiple parts (an example of multiple objects) are attached to device 140 (an example of a structure) captured by imaging device 110, and the device is transported by belt conveyor 150. In FIG. 1, image data 160 shows an example of one frame of image data included in the video data, and the example of FIG. 1 shows four parts (parts 161 to 164) attached to device 140. In this embodiment, parts 161 and 162 are parts to be inspected (an example of an object to be inspected), and parts 163 and 164 are parts other than the part to be inspected (an example of an object other than the part to be inspected).

[0012] The encoding device 120 encodes the image data of each frame included in the video data to generate encoded data. When generating the encoded data, the encoding device 120 encodes the image data using a compression rate map (a map indicating the compression rate for each region when the image data is encoded at a different compression rate for each region) transmitted from the image analysis device 130. The encoding device 120 also transmits the generated encoded data to the image analysis device 130 via the network 180.

[0013] Image analysis device 130 decodes the coded data transmitted from coding device 120 via network 180 to generate decoded data. Image analysis device 130 also identifies the regions of parts 161 and 162 (one example of multiple object regions, hereinafter referred to as multiple "part regions") by analyzing the generated decoded data (for example, by performing image recognition processing on the decoded data). Image analysis device 130 also identifies a reference region (here, the region on the outer edge of device 140) from the part region based on information indicating the configuration of device 140 (for example, design data). Furthermore, image analysis device 130 generates a compression rate map in which the compression rates of the part region and reference region are lower than the compression rates of regions other than the part region and reference region, and transmits the compression rate map to coding device 120.

[0014] This allows the encoding device 120 to encode image data using a compression rate map in which the compression rates for component regions and reference regions are lower than the compression rates for regions other than the component regions and reference regions. As a result, the amount of encoded data transmitted to the image analysis device 130 can be reduced compared to when image data is encoded using a compression rate map in which the compression rates for all regions are lowered.

[0015] Furthermore, image analysis device 130 decodes the encoded data with the reduced data volume to generate decoded data. At this time, image analysis device 130 can generate decoded data with high image quality for the component area and reference area. Therefore, by analyzing the decoded data (for example, by performing a contour extraction process on the decoded data), image analysis device 130 can extract the contour lines of component 161 and component 162 and the reference line (here, the contour line of the outer edge of device 140) with high accuracy. As a result, the positions of component 161 and component 162 based on the reference line can be measured with high accuracy.

[0016] In FIG. 1, position measurement data 170 shows that the positions of parts 161 and 162 based on the reference line are measured with high accuracy by calculating the distance (arrow) of part 161 from the reference line and the distance (arrow) of part 152 from the reference line.

[0017] In this way, according to the image processing system 100, when a part to be inspected is photographed as a moving image and analyzed by the image analysis device 130 at the destination, it is possible to reduce the amount of data of the moving image data to be transmitted while achieving highly accurate position measurement of the part.

[0018] <Hardware configuration of the encoding device and image analysis device> Next, a description will be given of the hardware configuration of the encoding device 120 and the image analysis device 130. Fig. 2 is a diagram showing an example of the hardware configuration of the encoding device and the image analysis device.

[0019] 2A is a diagram showing an example of the hardware configuration of an encoding device 120. The encoding device 120 includes a processor 201, a memory 202, an auxiliary storage device 203, an I / F (Interface) device 204, a communication device 205, and a drive device 206. The hardware components of the encoding device 120 are connected to each other via a bus 207.

[0020] The processor 201 has various arithmetic devices such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc. The processor 201 reads various programs (for example, an encoding program, etc.) into the memory 202 and executes them.

[0021] The memory 202 has a main storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The processor 201 and the memory 202 form a so-called computer, and the processor 201 executes various programs read onto the memory 202, causing the computer to realize various functions.

[0022] The auxiliary storage device 203 stores various programs and various data used when the processor 201 executes the various programs.

[0023] The I / F device 204 is a connection device that connects the image capture device 110, which is an example of an external device, with the encoding device 120.

[0024] The communication unit 205 is a communication device for communicating with the image analysis unit 130 via the network 180 .

[0025] The drive device 206 is a device for loading a recording medium 210. The recording medium 210 here includes media that record information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, a magneto-optical disk, etc. The recording medium 210 may also include semiconductor memory that records information electrically, such as a ROM, a flash memory, etc.

[0026] The various programs to be installed in the auxiliary storage device 203 are installed, for example, by setting the distributed recording medium 210 in the drive device 206 and reading the various programs recorded on the recording medium 210 by the drive device 206. Alternatively, the various programs to be installed in the auxiliary storage device 203 may be installed by being downloaded from the network 180 via the communication device 205.

[0027] 2B is a diagram showing an example of the hardware configuration of the image analysis device 130. Note that the hardware configuration of the image analysis device 130 is generally the same as the hardware configuration of the encoding device 120, and therefore the following description will focus on the differences from the encoding device 120.

[0028] The processor 221 reads, for example, an image analysis program or the like onto the memory 222 and executes it.

[0029] The I / F device 224 accepts operations for the image analysis device 130 via an operation device 231. The I / F device 224 also outputs the results of processing by the image analysis device 130 and displays them via a display device 232. The communication device 225 also communicates with the encoding device 120 via the network 180.

[0030] <Functional configuration of the encoding device and the image analysis device> Next, a description will be given of the functional configurations of the encoding device 120 and the image analysis device 130 of the image processing system 100 according to the first embodiment. Fig. 3 is a diagram showing an example of the functional configurations of the encoding device and the image analysis device of the image processing system according to the first embodiment.

[0031] As described above, the encoding device 120 has an encoding program installed therein, and the encoding device 120 functions as the encoding unit 121 and the compression rate setting unit 122 by executing the program.

[0032] The encoding unit 121 generates encoded data by encoding image data of each frame included in the video data based on the compression rate map set by the compression rate setting unit 122. The encoding unit 121 also transmits the generated encoded data to the image analysis device 130.

[0033] The compression rate setting unit 122 receives the compression rate map sent from the image analysis device 130 and sets it in the encoding unit 121 .

[0034] As described above, an image analysis program is installed in the image analysis device 130. When the program is executed, the image analysis device 130 functions as a decoding unit 131, a region identification unit 132_1, a contour extraction unit 132_2, a specification unit 133, and a compression ratio map generation unit 134.

[0035] The decoding unit 131 decodes the coded data transmitted from the coding device 120 to generate decoded data, and notifies the generated decoded data to the region identification unit 132_1 and the contour extraction unit 132_2.

[0036] The area identification unit 132_1 is an example of an identification unit. The area identification unit 132_1 has a trained deep learning model that has been machine-trained to identify various parts, etc. attached to the device 140 by performing image recognition processing on the decoded data. The area identification unit 132_1 inputs the decoded data into the trained deep learning model to identify various parts, etc., and notifies the identification unit 133 of the areas of the parts 161 and 162 to be inspected.

[0037] The identifying unit 133 identifies the areas of the inspection target components 161 and 162 identified by the area identifying unit 132_1 as component areas. Furthermore, the identifying unit 133 identifies a reference area (an area on the outer edge of the device 140) from the identified component areas based on design data 135 indicating the configuration of the device 140. Furthermore, the identifying unit 133 notifies the compression ratio map generating unit 134 and the contour extracting unit 132_2 of the identified component areas and reference area.

[0038] The compression rate map generation unit 134 has a default compression rate map, and when notified of the component area and reference area by the identification unit 133, generates a compression rate map in which the compression rates of the component area and reference area are changed to compression rates lower than the default compression rate. The compression rate map generation unit 134 also transmits the generated compression rate map to the encoding device 120.

[0039] The contour extraction unit 132_2 is an example of a measurement unit. The contour extraction unit 132_2 corrects the shape of the device 140 in the decoded data based on design data 135 that indicates the configuration of the device 140. Specifically, if the imaging direction when the imaging device 110 images the device 140 is tilted with respect to the vertical direction, for example, distortion occurs in the shape of the device 140 in the decoded data. Therefore, the contour extraction unit 132_2 corrects this distortion based on the design data 135. Furthermore, the contour extraction unit 132_2 enlarges or reduces the size of the device 140 in the decoded data so that the size of the device 140 in the decoded data matches the size of the device 140 indicated in the design data 135.

[0040] Here, the shape and size (hereinafter referred to as the shape, etc.) of the device 140 in the decoded data are corrected to match the design data 135, but the shape, etc. of the design data 135 may also be corrected to match the device 140 in the decoded data.

[0041] The contour extraction unit 132_2 also performs contour extraction processing on the decoded data in which the shape, etc. of the device 140 has been corrected, extracting contours to generate contour line image data. The contour extraction unit 132_2 also refers to the component area and reference area identified by the identification unit 133, and identifies the contour lines of the components 161 and 162 to be inspected and the reference line (the contour line of the outer edge of the device 140) from the contour lines in the contour line image data.

[0042] Furthermore, the contour extraction unit 132_2 calculates the distance of the contour line of the part 161 from the reference line and the distance of the contour line of the part 162 from the reference line, respectively, to measure the positions of the parts 161 and 162 based on the reference line, and outputs position measurement data 170.

[0043] <Examples of image processing using an image processing system> Next, a specific example of image processing by the image processing system 100 according to the first embodiment will be described. Figures 4 to 7 are first to fourth diagrams showing a specific example of image processing by the image processing system according to the first embodiment.

[0044] 4, when image data 160 is input, the encoding unit 121 encodes the image data 160 using a compression rate map 400 in which a default compression rate is set, to generate encoded data 410. The encoding unit 121 also transmits the encoded data 410 to the image analysis device 130.

[0045] The coded data 410 transmitted to the image analysis device 130 is decoded by the decoding unit 131 to generate decoded data 420. At this time, the decoded data 420 generated does not have sufficient image quality to perform highly accurate position measurement, but has image quality that allows parts 161 and 162 to be identified.

[0046] For this reason, by performing image recognition processing on the generated decoded data 420, the area identification unit 132_1 identifies the areas of the parts 161 and 162. Furthermore, the identification unit 133 identifies the areas of the parts 161 and 162 as part areas. In the decoded data 430, the dotted line 431 indicates the part area identified by the identification unit 133.

[0047] 5, the identification unit 133 identifies a reference area (an area on the outer edge of the device 140) from the component area based on the design data 135. In the decoded data 510 of FIG. 5, the area between the dotted line 511 and the dotted line 512 indicates the reference area identified by the identification unit 133.

[0048] The component region and reference region identified by the identification unit 133 are notified to the compression ratio map generation unit 134. The compression ratio map generation unit 134 generates a compression ratio map 500 by setting the compression ratios of the component region and reference region to a lower compression ratio than the default compression ratio in the compression ratio map 400, in which a default compression ratio is set.

[0049] Subsequently, when image data 160' of the frame next to image data 160 included in the video data is input, encoding unit 121 encodes image data 160' using compression rate map 500 to generate encoded data 520. Furthermore, encoding unit 121 transmits encoded data 520 to image analysis device 130.

[0050] 6, the coded data 520 transmitted to the image analysis device 130 is decoded by the decoding unit 131 to generate the decoded data 610. At this time, the generated decoded data 610 has sufficient image quality to perform high-precision position measurement for the component region and the reference region.

[0051] Therefore, by performing image recognition processing on the generated decoded data 610, the area identification unit 132_1 identifies the areas of the components 161 and 162 with high accuracy, and the identification unit 133 identifies the component areas. Furthermore, the identification unit 133 identifies a reference area from the component areas based on the design data 135.

[0052] 6, dotted line 621 indicates the part area identified by the identification unit 133 after the areas of parts 161 and 162 are identified with high accuracy by the area identification unit 132_1. Also, in decoded data 630 in FIG. 6, the area between dotted line 631 and dotted line 632 indicates the reference area identified by the identification unit 133.

[0053] Furthermore, for the generated decoded data 610, the contour extraction unit 132_2 corrects the shape and the like of the device 140 in the decoded data 610 based on the design data 135. In Fig. 6, the decoded data 640 is decoded data including the device 140' whose shape and the like have been corrected.

[0054] The contour extraction unit 132_2 extracts contours by performing contour extraction processing on the decoded data 640, and generates contour image data 650.

[0055] 7, the contour extraction unit 132_2 identifies four reference lines that are the contour lines of the outer edge of the device 140 from the contour lines in the contour line image data 650 by referring to the reference area identified by the identification unit 133. Note that the contour line image data 710 in FIG. 7 indicates, with rectangular dotted lines, four positions where the identified four reference lines are connected.

[0056] Next, the contour extraction unit 132_2 generates coordinates based on the identified reference line and the design data 135 (see the contour image data 720). Furthermore, the contour extraction unit 132_2 identifies the contour lines of the component 161 and the component 162 in the contour image data 720 by referring to the component areas identified by the identification unit 133. Furthermore, the contour extraction unit 132_2 calculates the distance from the identified reference line for each of the identified contour lines of the component 161 and the component 162, based on the generated coordinates.

[0057] The arrows shown in the contour line image data 730 indicate the calculated distances of the contour lines of the component 161 and the component 162 from the reference line.

[0058] <Image processing flow by image processing system> Next, a description will be given of the flow of image processing by the image processing system 100 according to the first embodiment. Figures 8 and 9 are first and second flowcharts showing the flow of image processing by the image processing system according to the first embodiment.

[0059] In step S801, the encoding device 120 acquires one frame of image data (an example of first image data) from the video data captured by the imaging device 110.

[0060] In step S802, the encoding device 120 generates encoded data by encoding the acquired image data using a default compression rate map, and transmits the encoded data to the image analysis device 130 via the network 180.

[0061] In step S803, the image analyzing device 130 receives the coded data and decodes the received coded data to generate decoded data (an example of first decoded data).

[0062] In step S804, the image analysis device 130 performs image recognition processing on the decoded data to identify the area of ​​the part to be inspected.

[0063] In step S805, the image analyzing device 130 identifies the identified area of ​​the component to be inspected as a component area, and also identifies a reference area from the component area by referring to the design data.

[0064] In step S806, the image analysis device 130 generates a compression rate map by setting the compression rates of the identified component area and reference area to a compression rate (second compression rate) lower than the default compression rate (first compression rate).

[0065] In step S807, image analyzing device 130 determines whether to proceed to the measurement phase. If the component area cannot be identified, image analyzing device 130 determines not to proceed to the measurement phase (determines NO in step S807) and returns to step S801. In this case, steps S801 to S806 are executed for the next frame of image data.

[0066] On the other hand, if the component area has been identified, it is determined to proceed to the measurement phase (YES in step S807), and the process proceeds to step 901 in Fig. 9. In this case, the compression ratio map generated in step S806 is transmitted to the encoding device 120.

[0067] In step S901, the encoding device 120 obtains the next frame of image data (an example of second image data).

[0068] In step S902, the encoding device 120 generates encoded data by encoding the acquired image data using the compression rate map generated in step S806, and transmits the encoded data to the image analysis device .

[0069] In step S903, the image analyzing device 130 receives the coded data and decodes the received coded data to generate decoded data (second decoded data).

[0070] In step S904, the image analysis device 130 performs image recognition processing on the decoded data to identify the area of ​​the component to be inspected and specify the component area. The image analysis device 130 also specifies a reference area based on the specified component area and the design data.

[0071] In step S905, the image analysis device 130 corrects the shape and the like of the device 140 in the decoded data using the design data.

[0072] In step S906, the image analysis device 130 extracts contours and generates contour image data by performing a contour extraction process on the decoded data in which the shape, etc., of the device 140 has been corrected. Furthermore, the image analysis device 130 identifies a reference line from among the contours in the contour image data by referring to the reference area identified in step S904.

[0073] In step S907, the image analysis device 130 generates coordinates based on the identified reference line.

[0074] In step S908, image analyzing device 130 identifies the contours of inspection target components 161 and 162 from the contours in the contour image data by referring to the component regions identified in step S904. Furthermore, image analyzing device 130 calculates the distances from the reference line to the contours of inspection target components 161 and 162 based on the generated coordinates, and measures the positions of inspection target components 161 and 162.

[0075] In step S909, the image analysis device 130 determines whether or not to end the image processing, and if it determines that the image processing should be continued (NO in step S909), the process proceeds to step S910.

[0076] In step S910, image analyzing device 130 determines whether or not switching to the next device has occurred. If it is determined in step S910 that switching to the next device has not occurred (NO in step S910), the process returns to step S901.

[0077] On the other hand, if it is determined in step S910 that the device has been switched to the next device (YES in step S910), the process returns to step S801 in FIG.

[0078] Furthermore, in step S909, if it is determined that the image processing is to be ended (YES in step S909), the image processing is ended.

[0079] <Example of changes in compression ratio> Next, a description will be given of changes in the compression ratios set for the component area and the reference area during image processing by the image processing system 100 according to the first embodiment. Fig. 10 is a first diagram showing an example of changes in the compression ratio.

[0080] 10, reference numeral 1010 denotes image data of each frame included in moving image data captured by the imaging device 110. In the example of FIG. 10, the moving image data indicated by reference numeral 1010 includes: Image data taken by a device with identification number 001, Image data taken during the switchover from device with ID number 001 to device with ID number 002, Image data taken by a device with identification number 002, indicates that it contains

[0081] In Fig. 10, reference numeral 1020 indicates the main content of image processing performed on the image data indicated by reference numeral 1010. According to the example of Fig. 10, when a device with identification number = 001 is included in the shooting range, the image processing system 100 performs image recognition processing to identify the component area and the reference area, and generates and sets a compression rate map (time t1). At this time, the compression rates set for the component area and the reference area are lower than the default compression rate, as indicated by reference numeral 1020.

[0082] Furthermore, when the position measurement of the part is completed, the image processing system 100 sets a default compression rate (time t2).

[0083] Subsequently, when a device with identification number=002 is included in the photographing range, the image processing system 100 performs image recognition processing to identify the component area and the reference area, and generates and sets a compression rate map (time t3).

[0084] Furthermore, when the position measurement of the part is completed, the image processing system 100 sets a default compression ratio (time t4). Note that thereafter, the same image processing is repeated, and the same changes in the compression ratio are also repeated.

[0085] As is clear from the above description, the image processing system 100 according to the first embodiment decodes first image data encoded at a default compression rate, and upon acquiring the first decoded data, performs image recognition processing to identify the region of the component to be inspected. Furthermore, the image processing system 100 according to the first embodiment identifies the identified region of the component to be inspected as a component region, and identifies a reference region containing a reference line using the identified component region and device design data. Furthermore, the image processing system 100 according to the first embodiment generates a compression rate map in which the compression rates of the component region and the reference region are changed to a compression rate lower than the default compression rate, and encodes the second image data using the generated compression rate map. Furthermore, upon acquiring second decoded data obtained by decoding the encoded second image data, the image processing system 100 according to the first embodiment performs contour extraction processing to identify the contour line and reference line of the component to be inspected, thereby measuring the position of the component to be inspected based on the reference line.

[0086] As a result, according to the image processing system 100 of the first embodiment, when a part to be inspected is photographed as a moving image and analyzed by an image analysis device to which the image data is transmitted, the amount of data of the moving image data to be transmitted can be reduced.

[0087] [Second embodiment] In the first embodiment, a component area is first identified, and then a reference area is identified using the identified component area and design data. However, the method of identifying the component area and reference area is not limited to this. For example, the reference area may be identified first, and then the component area may be identified using the identified reference area and design data. The second embodiment will be described below, focusing on the differences from the first embodiment.

[0088] <Examples of image processing using an image processing system> First, a specific example of image processing by the image processing system 100 according to the second embodiment will be described. Figures 11 to 14 are first to fourth diagrams showing a specific example of image processing by the image processing system according to the second embodiment, and correspond to Figures 4 to 7 described in the first embodiment.

[0089] 11, a specific example of the encoding and decoding process is the same as the specific example of the encoding and decoding process in Fig. 4, and therefore a description thereof will be omitted here. In this embodiment, the decoded data 420 generated by the decoding unit 131 of the image analysis device 130 does not have sufficient image quality to perform highly accurate position measurement, but has image quality sufficient to identify the outer edge of the device 140.

[0090] For this reason, by performing image recognition processing on the generated decoded data 420, the area identification unit 132_1 identifies the area on the outer edge of the device 140. Furthermore, the identification unit 133 identifies the area on the outer edge of the device 140 as a reference area. In the decoded data 1110, the area between the dotted line 1111 and the dotted line 1112 indicates the reference area identified by the identification unit 133.

[0091] 12, the identification unit 133 identifies a component area from the reference area based on the design data 135. In the decoded data 510 of FIG. 12, a dotted line 1211 indicates the component area identified by the identification unit 133.

[0092] Note that the subsequent processes shown in Fig. 12 (the process of generating the compression rate map 500 and the encoding process for the image data 160' of the next frame) are the same as the corresponding processes shown in Fig. 5, and therefore their explanations will be omitted here. Also, the decoding process shown in Fig. 13 is the same as the decoding process shown in Fig. 6, and therefore their explanations will be omitted here.

[0093] 13, the area identification unit 132_1 in this embodiment identifies the outer edge area of ​​the device 140 with high accuracy, and the identification unit 133 identifies the reference area. Furthermore, the identification unit 133 identifies the component area from the reference area based on the design data 135.

[0094] In the decoded data 1310 of FIG. 13, the area between the dotted lines 1311 and 1312 indicates the reference area identified by the identification unit 133 as a result of the area identification unit 132_1 identifying the outer edge area of ​​the device 140 with high accuracy.

[0095] In the decoded data 1320 of FIG. 13, a dotted line 1321 indicates a component area identified by the identifying unit 133.

[0096] In addition, in FIG. 13, the shape correction processing and the contour extraction processing executed by the contour extraction unit 132_2 have already been explained in FIG. 6, and therefore explanation thereof will be omitted here.

[0097] 14, the contour extraction unit 132_2 refers to the reference area identified by the identification unit 133, and identifies four reference lines that are the contour lines of the outer edge of the device 140 from the contour lines in the contour line image data 650. Note that the contour line image data 710 in FIG. 14 indicates, with rectangular dotted lines, four positions where the identified four reference lines are connected.

[0098] Next, the contour extraction unit 132_2 generates coordinates based on the identified reference line and the design data 135 (see the contour image data 720). Furthermore, the contour extraction unit 132_2 identifies the contour lines of the component 161 and the component 162 in the contour image data 720 by referring to the component areas identified by the identification unit 133. Furthermore, the contour extraction unit 132_2 calculates the distance from the identified reference line for each of the identified contour lines of the component 161 and the component 162, based on the generated coordinates.

[0099] The arrows shown in the contour line image data 730 indicate the calculated distances from the reference line to the contour lines of the component 161 and the contour lines of the component 162, respectively.

[0100] <Image processing flow by image processing system> Next, the flow of image processing by the image processing system 100 according to the second embodiment will be described. Figures 15 and 16 are first and second flowcharts showing the flow of image processing by the image processing system according to the second embodiment. The differences from Figures 8 and 9 are steps S1501, S1502, S1601, S1602, and S1603.

[0101] In step S1501, the image analysis device 130 performs image recognition processing on the decoded data to identify the outer edge area of ​​the device 140 and specify the reference area.

[0102] In step S1502, the image analyzing device 130 identifies a component area based on the identified reference area and the design data.

[0103] In step S1601, the image analysis device 130 performs image recognition processing on the decoded data to identify the outer edge area of ​​the device 140 and specify a reference area. The image analysis device 130 also specifies a component area based on the specified reference area and design data.

[0104] In step S1602, the image analysis device 130 extracts contours and generates contour image data by performing contour extraction processing on the decoded data in which the shape, etc., of the device 140 has been corrected. Furthermore, the image analysis device 130 identifies a reference line from among the contours in the contour image data by referring to the reference area identified in step S1601.

[0105] In step S1603, image analyzing device 130 refers to the component area identified in step S1601 to identify the contours of inspection target components 161 and 162 from the contours in the contour image data. Furthermore, image analyzing device 130 calculates the distance from the reference line to the contours of inspection target components 161 and 162 based on the generated coordinates, and measures the positions of inspection target components 161 and 162.

[0106] As is clear from the above description, the image processing system 100 according to the second embodiment decodes first image data encoded at a default compression rate, and upon acquiring the first decoded data, performs image recognition processing to identify the peripheral region of the device. Furthermore, the image processing system 100 according to the second embodiment identifies the identified peripheral region of the device as a reference region, and identifies a component region containing a component to be inspected using the identified reference region and the design data of the device. Furthermore, the image processing system 100 according to the second embodiment generates a compression rate map in which the compression rates of the component region and the reference region are changed to a compression rate lower than the default compression rate, and encodes the second image data using the generated compression rate map. Furthermore, upon acquiring second decoded data obtained by decoding the encoded second image data, the image processing system 100 according to the second embodiment performs contour extraction processing to identify the contour line and reference line of the component to be inspected, thereby measuring the position of the component to be inspected based on the reference line.

[0107] As a result, according to the image processing system 100 of the second embodiment, when a part to be inspected is photographed as a moving image and analyzed by an image analysis device to which the image data is transmitted, the amount of data of the moving image data to be transmitted can be reduced.

[0108] [Third embodiment] In the first embodiment, the area identification unit 132_1 identifies the parts to be inspected in the first decoded data and the second decoded data. In contrast, in the third embodiment, a first area identification unit and a second area identification unit are provided instead of the area identification unit 132_1. The first area identification unit identifies the parts to be inspected from other types of parts in the first decoded data, and the second area identification unit individually identifies the parts to be inspected in the second decoded data. The third embodiment will be described below, focusing on the differences from the first and second embodiments.

[0109] <Functional configuration of the encoding device and the image analysis device> First, a description will be given of the functional configurations of the encoding device and image analysis device of the image processing system 100 according to the third embodiment. Fig. 17 is a diagram showing an example of the functional configurations of the encoding device and image analysis device of the image processing system according to the third embodiment.

[0110] The functional configuration differs from that described with reference to FIG. 3 in that the image analyzing device 130 has a first area classification section 1710_1 and a second area classification section 1710_2 instead of the area classification section 132_1.

[0111] The first area identification unit 1710_1 is an example of a first identification unit. The first area identification unit 1710_1 has a trained deep learning model that has been machine-learned to distinguish components to be inspected that are attached to the device 140 from components other than those to be inspected by performing image recognition processing on the first decoded data. The first area identification unit 1710_1 inputs the first decoded data to the trained deep learning model, thereby identifying the area of ​​the component to be inspected and notifying the identification unit 133.

[0112] The second area identification unit 1710_2 is an example of a second identification unit. The second area identification unit 1710_2 has a trained deep learning model that has been machine-trained to individually identify each of the inspection target components attached to the device 140 by performing image recognition processing on the second decoded data. The second area identification unit 1710_2 inputs the second decoded data to the trained deep learning model, thereby individually identifying the regions of the inspection target components and notifying the identification unit 133.

[0113] Note that, for example, when a plurality of parts to be inspected are attached to the device 140 and are the same type but have different performance, the second area identification unit 1710_2 can identify the individual parts. Therefore, for example, the second area identification unit 1710_2 can inspect whether or not the plurality of parts to be inspected are attached in attachment positions according to their performance.

[0114] <Examples of image processing using an image processing system> Next, a specific example of image processing by the image processing system 100 according to the third embodiment will be described. Figures 18 to 21 are first to fourth diagrams showing a specific example of image processing by the image processing system according to the third embodiment, and correspond to Figures 4 to 7 described in the first embodiment.

[0115] In FIG. 18, the encoding and decoding processes and the process of identifying component areas are the same as the specific example of the encoding and decoding processes and the process of identifying component areas in FIG. 4, and therefore a description thereof will be omitted here.

[0116] 18, first area identification unit 1710_1 identifies that inspection target components 161 and 162 are different from inspection target components 163 and 164. Identification unit 133 identifies the areas identified as inspection target components 161 and 162 as component areas. In decoded data 1810, dotted lines 1811 indicate component areas identified by identification unit 133.

[0117] 19, the identification unit 133 identifies a reference area (an area on the outer edge of the device 140) from the component area based on the design data 135. In the decoded data 510 of FIG. 19, the area between the dotted line 511 and the dotted line 512 indicates the reference area identified by the identification unit 133.

[0118] Note that the subsequent processes shown in Fig. 19 (the process of generating the compression rate map 500 and the process of encoding the image data 160' of the next frame) are the same as the corresponding processes shown in Fig. 5, and therefore their explanations will be omitted here. Also, the decoding process and the process of identifying the component area shown in Fig. 20 are the same as the decoding process and the process of identifying the component area shown in Fig. 6, and therefore their explanations will be omitted here.

[0119] 20 , second area identification unit 1710_2 identifies that inspection target components 161 and 162 are different types of components from components 163 and 164 not being inspection targets, and also identifies that components 161 and 162 have different performance characteristics. Identification unit 133 then distinguishes and identifies individual component areas. In decoded data 2010, dotted line 2011 indicates that identification unit 133 has identified a component area of ​​inspection target component 161, and dotted line 2012 indicates that identification unit 133 has identified a component area of ​​inspection target component 162. Identification unit 133 also identifies a reference area from the component areas based on design data 135. In decoded data 2020 in FIG. 20 , the area between dotted line 2021 and dotted line 2022 indicates the reference area identified by identification unit 133.

[0120] In addition, in FIG. 20, the shape correction processing and the contour extraction processing executed by the contour extraction unit 132_2 have already been explained in FIG. 6, and therefore explanation thereof will be omitted here.

[0121] Next, as shown in FIG. 21, the contour extraction unit 132_2 refers to the reference area identified by the identification unit 133 to identify four reference lines that are the contour lines of the outer edge of the device 140 from the contour lines in the contour line image data 650.

[0122] 21 (coordinate generation processing, position measurement processing) are the same as the corresponding processing shown in FIG. 7, and therefore description thereof will be omitted here. In the case of FIG. 21, however, the contour extraction unit 132_2 refers to the component area identified by the identification unit 133. Therefore, according to the contour extraction unit 132_2 in this embodiment, in addition to being able to measure the position of the component to be inspected, it is also possible to inspect, for example, whether or not each of the multiple components to be inspected is attached at an attachment position according to its performance.

[0123] As is clear from the above description, the image processing system 100 according to the third embodiment is provided with a first area classification unit 1710_1 and a second area classification unit 1710_2 instead of the area classification unit 132_1. As a result, according to the third embodiment, it is possible to achieve the same effects as the first and second embodiments and to perform inspections other than position measurement.

[0124] [Fourth embodiment] In the first embodiment, the case where various parts and the like attached to the device 140 are identified by performing image recognition processing has been described.

[0125] However, instead of performing image recognition processing, the area of ​​the component to be inspected may be identified by performing processing to calculate the difference in pixel values ​​of image data between frames of moving image data, for example.

[0126] In the second embodiment, the case where the outer edge area of ​​the device 140 is identified by performing image recognition processing has been described.

[0127] However, instead of performing image recognition processing, the outer edge area of ​​the device may be identified by, for example, performing processing to calculate the difference in pixel values ​​of image data between frames of moving image data. The following describes the fourth embodiment, focusing on the differences from the first and second embodiments.

[0128] <Functional configuration of the encoding device and the image analysis device> First, a description will be given of the functional configurations of the encoding device and image analysis device of the image processing system 100 according to the fourth embodiment. Fig. 22 is a diagram showing an example of the functional configurations of the encoding device and image analysis device of the image processing system according to the fourth embodiment.

[0129] The difference from FIG. 3 is that the image analysis device 130 has an image processing unit 2210 instead of the region identification unit 132_1.

[0130] The image processing unit 2210 calculates the difference in pixel values ​​of image data between frames of the moving image data. For example, if the colors of the parts 161 and 162 are different from the color of the device 140, the image processing unit 2210 calculates the difference in pixel values ​​of image data between frames of the moving image data. Image data before the parts 161 and 162 are attached to the device 140; Image data after the parts 161 and 162 are attached to the device 140; The image processing unit 2210 detects an area where the difference in pixel values ​​between the image data and the component area is equal to or greater than a predetermined threshold, and notifies the identification unit 133. As a result, the identification unit 133 can identify the area notified by the image processing unit 2210 as a component area. In this case, the identification unit 133 identifies a reference area from the identified component area based on the design data 135.

[0131] Alternatively, if the color of the device 140 is different from the color of the belt conveyor 150, the image processing unit 2210 Image data before the device 140 is placed on the belt conveyor 150; Image data after the device 140 is placed on the belt conveyor 150; The image processing unit 2210 detects an area where the difference in pixel values ​​between the image data and the component area is equal to or greater than a predetermined threshold, and notifies the identification unit 133. As a result, the identification unit 133 can identify the area notified by the image processing unit 2210 as the reference area. In this case, the identification unit 133 identifies a component area from the identified reference area based on the design data 135.

[0132] As is clear from the above description, the image processing system 100 according to the fourth embodiment has an image processing unit that calculates the difference in pixel values ​​of image data between frames of moving image data, instead of the region identification unit 132_1. As a result, according to the fourth embodiment, it is possible to identify a component region or a reference region, thereby achieving the same effects as those of the first or second embodiment.

[0133] [Fifth embodiment] In the first to fourth embodiments, the compression ratios of the component area and the reference area are changed in accordance with the progress of image processing (see FIG. 10). However, the timing of changing the compression ratios of the component area and the reference area is not limited to this. The fifth embodiment will be described below, focusing on the differences from the above embodiments.

[0134] <Example of changes in compression ratio> Fig. 23 is a second diagram showing an example of a change in compression rate. In Fig. 23, reference numeral 2310 indicates image data of each frame included in moving image data captured by the imaging device 110. As in Fig. 10, the example in Fig. 23 shows that the moving image data indicated by reference numeral 2310 is Image data taken by a device with identification number 001, Image data taken during the switchover from device with ID number 001 to device with ID number 002, Image data taken by a device with identification number 002, indicates that it contains

[0135] In Fig. 23, reference numeral 2320 indicates the state of the device 140 while the moving image data is being captured. According to the example of Fig. 23, first, a device with identification number = 001, which has no component to be inspected attached thereto, is placed on the belt conveyor 150. Next, a component to be inspected is attached to the device with identification number = 001 placed on the belt conveyor 150, and then the device is moved to the next process.

[0136] Next, a device with identification number = 002, which does not have a component to be inspected attached, is placed on the belt conveyor 150. Next, a component to be inspected is attached to the device with identification number = 002 placed on the belt conveyor 150.

[0137] 23, reference numeral 2330 indicates the main content of image processing performed on the image data indicated by reference numeral 2310. According to the example of Fig. 23, when a device with identification number = 001 is placed on the belt conveyor 150 and a part to be inspected is attached, the image processing system 100 detects the device to which the part is attached.

[0138] Furthermore, the image processing system 100 performs image recognition processing to identify component regions and reference regions, and generates and sets a compression rate map. Furthermore, the image processing system 100 measures the positions of the components.

[0139] Furthermore, when a device with identification number=002 is placed on the belt conveyor 150 and a part to be inspected is attached, the image processing system 100 detects the device to which the part is attached.

[0140] Furthermore, the image processing system 100 performs image recognition processing to identify component regions and reference regions, and generates and sets a compression rate map. Furthermore, the image processing system 100 measures the positions of the components.

[0141] In contrast to this, in the image processing system 100 according to the fifth embodiment, as indicated by the reference numeral 2340, the compression rates of the component area and the reference area are changed as follows. The maximum compression ratio (an example of the third compression ratio) is set until the component to be inspected is attached to the device placed on the belt conveyor 150 (for example, until time t1). When the part to be inspected is installed, a default compression ratio is set (time t1). When the image recognition process is performed and the component area and the reference area are identified, a compression rate lower than the default compression rate is set (time t2). Once the position measurement for the part to be inspected is completed, the maximum compression ratio is set (time t3). After the device is moved by the belt conveyor 150, the maximum compression ratio is set until the next device is placed and the component to be inspected is attached (from time t3 to time t4). Once the part to be inspected is installed, the default compression ratio is set (time t4). When the image recognition process is performed and the component area and the reference area are identified, a compression rate lower than the default compression rate is set (time t5). Once the position measurement is completed for the part to be inspected, the maximum compression ratio is set (time t6).

[0142] In this way, the image processing system 100 according to the fifth embodiment changes the compression rate according to the progress of image processing, and also changes the compression rate according to the state of the device. As a result, according to the fifth embodiment, the time for which a high compression rate is set can be extended, and therefore the amount of encoded data can be reduced when the encoded data is transmitted from the encoding device 120 to the image analyzing device 130.

[0143] In the above description, the maximum compression rate is set until time t1, or between time t3 and time t4, or after time t6. However, instead of setting the maximum compression rate, the transmission of encoded data to the image analysis device 130 may be stopped.

[0144] [Sixth embodiment] In the above embodiments, image processing has been described for the case where multiple parts to be inspected are attached to the device at the same time. On the other hand, multiple parts to be inspected may be attached to the device at different times. In the sixth embodiment, image processing will be described for the case where multiple parts to be inspected are attached to the device at different times.

[0145] <Example of changes in compression ratio> (1) When measuring the position of the part to be inspected each time it is installed Fig. 24 is a third diagram showing an example of a change in compression rate. In Fig. 24, reference numeral 2410 indicates image data of each frame included in moving image data captured by the imaging device 110. As in Fig. 23, the example in Fig. 24 shows that the moving image data indicated by reference numeral 2410 is Image data taken by a device with identification number 001, Image data taken during the switchover from device with ID number 001 to device with ID number 002, Image data taken by a device with identification number 002, indicates that it contains

[0146] 24, reference numeral 2420 indicates the state of the device while the moving image data is being captured. According to the example of Fig. 24, first, a device with identification number = 001, on which no component to be inspected is attached, is placed on belt conveyor 150. Next, components 1, 2, and 3 to be inspected are attached at different times to the device with identification number = 001 placed on belt conveyor 150, and then the device is moved to the next process.

[0147] Next, a device with identification number = 002, which does not have a component to be inspected attached, is placed on belt conveyor 150. Next, components 1, 2, and 3 to be inspected are attached to the device with identification number = 002 placed on belt conveyor 150 at different times.

[0148] 24, reference numeral 2430 indicates the main content of image processing performed on image data included in the moving image data indicated by reference numeral 2410. According to the example of Fig. 24, when a device with identification number = 001 is placed on the belt conveyor 150, the image processing system 100 detects the device.

[0149] Furthermore, the image processing system 100 performs image recognition processing to identify the component area and reference area of ​​the component 1 to be inspected, and generates and sets a compression rate map. After that, the image processing system 100 measures the position of the component 1 to be inspected.

[0150] Furthermore, the image processing system 100 performs image recognition processing to identify the component area and reference area of ​​the component 2 to be inspected, and generates and sets a compression rate map. Thereafter, the image processing system 100 measures the position of the component 2 to be inspected.

[0151] Furthermore, the image processing system 100 performs image recognition processing to identify the component area and reference area of ​​the component 3 to be inspected, and generates and sets a compression rate map. Thereafter, the image processing system 100 measures the position of the component 3 to be inspected.

[0152] Furthermore, when a device with identification number=002 is placed on the belt conveyor 150, the image processing system 100 detects the device.

[0153] Furthermore, the image processing system 100 performs image recognition processing to identify the component area and reference area of ​​the component 1 to be inspected, and generates and sets a compression rate map. After that, the image processing system 100 measures the position of the component 1 to be inspected.

[0154] Furthermore, the image processing system 100 performs image recognition processing to identify the component area and reference area of ​​the component 2 to be inspected, and generates and sets a compression rate map. Thereafter, the image processing system 100 measures the position of the component 2 to be inspected.

[0155] Furthermore, the image processing system 100 performs image recognition processing to identify the component area and reference area of ​​the component 3 to be inspected, and generates and sets a compression rate map. Thereafter, the image processing system 100 measures the position of the component 3 to be inspected.

[0156] In contrast to this, in the example of FIG. 24 of the image processing system 100 according to the sixth embodiment, as indicated by reference numeral 2440, the compression rates of the component areas and reference area of ​​components 1 to 3 to be inspected are changed as follows. A default compression ratio is set until the part 1 to be inspected is attached to the device placed on the belt conveyor 150 (for example, until time t1). When the component 1 to be inspected is attached and the component area and reference area of ​​the component 1 to be inspected are identified, a compression rate lower than the default compression rate is set for the identified area (time t1). After the position measurement of the inspection target component 1 is completed and the inspection target component 2 is attached, the component area and reference area of ​​the inspection target component 2 are identified. Then, a compression rate lower than the default compression rate is set for the identified area (time t2). After the position measurement of the inspection target component 2 is completed and the inspection target component 3 is attached, the component area and reference area of ​​the inspection target component 3 are identified. Then, a compression rate lower than the default compression rate is set for the identified area (time t3). When the position measurement for the inspected part 3 is completed, the maximum compression ratio is set (time t4). After the device is moved by the belt conveyor 150, the maximum compression ratio is set until the next device is placed (from time t4 to time t5). When the next device is placed on the belt conveyor 150, a default compression ratio is set until the component 1 to be inspected is attached (from time t5 to time t6). When the component 1 to be inspected is attached and the component area and reference area of ​​the component 1 to be inspected are identified, a compression rate lower than the default compression rate is set for the identified area (time t6). After the position measurement of the inspection target component 1 is completed and the inspection target component 2 is attached, the component area and reference area of ​​the inspection target component 2 are identified. Then, a compression rate lower than the default compression rate is set for the identified area (time t7). After the position measurement of the inspection target component 2 is completed and the inspection target component 3 is attached, the component area and reference area of ​​the inspection target component 3 are identified. Then, a compression rate lower than the default compression rate is set for the identified area (time t8).

[0157] 24 of the image processing system 100 according to the sixth embodiment changes the compression rate according to the progress of image processing, and also changes the compression rate according to the attachment of a part to the device. As a result, according to the sixth embodiment, position measurement can be performed every time a part is attached to the device.

[0158] (2) When measuring the position after the installation of all parts to be inspected is completed Fig. 25 is a fourth diagram showing an example of changes in compression rate. In Fig. 25, reference numeral 2510 indicates image data of each frame included in moving image data captured by imaging device 110. Also in Fig. 25, reference numeral 2520 indicates the state of device 140 while capturing the moving image data. Note that reference numerals 2510 and 2520 in Fig. 25 are the same as reference numerals 2410 and 2420 in Fig. 24, and therefore description thereof will be omitted here.

[0159] 25, reference numeral 2530 indicates the main content of image processing performed on image data included in the moving image data indicated by reference numeral 2510. According to the example of Fig. 25, when a device with identification number = 001 is placed on the belt conveyor 150, the image processing system 100 detects the device.

[0160] Furthermore, when the installation of the inspection target parts 1, 2, and 3 is completed, the image processing system 100 performs image recognition processing to identify the part areas and reference areas of the inspection target parts 1, 2, and 3, and generates and sets a compression rate map. After that, the image processing system 100 measures the positions of the inspection target parts 1, 2, and 3.

[0161] Furthermore, when a device with identification number=002 is placed on the belt conveyor 150, the image processing system 100 detects the device.

[0162] Furthermore, when the installation of the inspection target parts 1, 2, and 3 is completed, the image processing system 100 performs image recognition processing to identify the part areas and reference areas of the inspection target parts 1, 2, and 3, and generates and sets a compression rate map. After that, the image processing system 100 measures the positions of the inspection target parts 1, 2, and 3.

[0163] In contrast to this, in the example of FIG. 25 of the image processing system 100 according to the sixth embodiment, as indicated by reference numeral 2540, the compression rates of the component areas and reference area of ​​components 1 to 3 to be inspected are changed as follows: The maximum compression rate is set until the device is placed on the belt conveyor 150 (for example, until time t1), and once the device is placed on the belt conveyor 150, the default compression rate is set (time t1). After the installation of the inspection target parts 1, 2, and 3 is completed and the part areas and reference areas of the inspection target parts 1, 2, and 3 are identified, a compression rate lower than the default compression rate is set for the identified areas (time t2). When the position measurements for the inspected parts 1, 2, and 3 are completed, the maximum compression ratio is set (time t3). When the belt conveyor 150 moves the device and the next device is placed, a default compression ratio is set (time t4). After the installation of the inspection target parts 1, 2, and 3 is completed and the part areas and reference areas of the inspection target parts 1, 2, and 3 are identified, a compression rate lower than the default compression rate is set for the identified areas (time t5).

[0164] 25 of the image processing system 100 according to the sixth embodiment not only changes the compression rate as the image processing progresses, but also changes the compression rate when all of the components to be inspected have been installed in the device. As a result, according to the sixth embodiment, it is possible to shorten the time for which the compression rate is set low, and reduce the amount of data transmitted from the encoding device 120 to the image analyzing device 130.

[0165] [Other embodiments] In the above embodiments, when the compression ratios of the component area and the reference area are changed to a lower value than the default compression ratio, the compression ratios of the areas other than the component area and the reference area are maintained at the default compression ratio. However, the compression ratios of the areas other than the component area and the reference area do not need to be maintained at the default compression ratio, and may be changed to, for example, the maximum compression ratio.

[0166] In addition, in the above embodiments, the case where the peripheral region of the device is specified as the reference region has been described, but a region other than the peripheral region of the device may be specified as the reference region. Alternatively, the peripheral region of the device may be specified as the reference region, and a region other than the peripheral region of the device may be specified as the quasi-reference region. For example, an easily identifiable part attached to the device may be specified as the quasi-reference region.

[0167] In addition, in the above embodiments, the design data is used to identify the reference area from the identified component area, or to identify the component area from the identified reference area. However, information other than the design data may be used as long as it indicates the configuration of the device.

[0168] In the above embodiments, the component area and reference area identified by the identifying unit are referenced when identifying the contour line and reference line of the component to be inspected in the contour line image data. However, information other than the component area and reference area identified by the identifying unit may also be referenced. Examples of information other than the component area and reference area identified by the identifying unit include image features (color, shape, texture, etc.) of the component area to be inspected.

[0169] Furthermore, although the above embodiments do not mention details of the image data used for position measurement, the image data used for position measurement may be, for example, one frame of image data or multiple frames of image data. When multiple frames of image data are used, the most appropriate measurement result may be selected and output from multiple measurement results obtained by performing position measurement on each image data. Alternatively, the average value of multiple measurement results obtained by performing position measurement on each image data may be calculated and output.

[0170] In addition, in each of the above embodiments, the identification unit and the compression rate map generation unit are implemented in the image analysis device 130. However, the identification unit and the compression rate map generation unit may be implemented in the encoding device 120, for example.

[0171] The present invention is not limited to the configurations described in the above embodiments, but may be combined with other elements, etc. These aspects can be changed without departing from the spirit of the present invention, and can be appropriately determined depending on the application form. [Explanation of symbols]

[0172] 100: Image processing system 110: Imaging device 120: Encoding device 121: Encoding section 122: Compression rate setting section 130: Image analysis device 131: Decryption unit 132_1: Area identification section 132_2: Contour extraction section 133: Specific part 134: Compression ratio map generation unit 135: Design data 1710_1: First area identification unit 1710_2: Second area identification unit 2210: Image processing unit

Claims

1. an identification unit that decodes the first image data encoded at the first compression rate, and when acquiring first decoded data, analyzes the first decoded data to identify an object region including an object to be inspected and a reference region including a reference line; an identification unit that identifies the object region and the reference region using the identified region and information indicating the configuration of a structure to which the object to be inspected is attached; a measurement unit that generates a compression rate map in which the object region and the reference region are compressed at a second compression rate that is smaller than the first compression rate, and, when second decoded data is obtained by decoding second image data that has been coded using the compression rate map, measures the position of the object to be inspected based on the reference line by analyzing the second decoded data; An image analysis device having the above.

2. the identification unit identifies a region of the object to be inspected by performing image recognition processing on the first decoded data; 2. The image analysis device according to claim 1, wherein the identification unit identifies a region of the identified object to be inspected as the object region, and identifies the reference region using the identified object region and information indicating the configuration of the structure.

3. the identification unit identifies an outer edge region of the structure by performing image recognition processing on the first decoded data; 2. The image analysis device according to claim 1, wherein the identification unit identifies an outer edge region of the identified structure as the reference region, and identifies the object region using the identified reference region and information indicating a configuration of the structure.

4. the identification unit identifies a region of the object to be inspected by performing image recognition processing on the second decoded data; the identification unit identifies a region of the object to be inspected identified in the second decoded data as the object region, and identifies the reference region in the second decoded data using the identified object region and information indicating a configuration of the structure; the measurement unit extracts a contour line from the second decoded data; Identifying a contour of the object by referencing the object region identified in the second decoded data; The image analyzing device according to claim 2 , wherein the contour line serving as the reference line is identified by referring to the reference area identified in the second decoded data.

5. the identification unit identifies an outer edge region of the structure by performing image recognition processing on the second decoded data; the identification unit identifies an outer edge region of the structure identified in the second decoded data as the reference region, and identifies the object region in the second decoded data using the identified reference region and information indicating a configuration of the structure; the measurement unit extracts a contour line from the second decoded data; Identifying a contour of the object by referencing the object region identified in the second decoded data; The image analyzing device according to claim 3 , wherein the contour line serving as the reference line is identified by referring to the reference area identified in the second decoded data.

6. The image analyzing device according to claim 1 , wherein the measuring unit corrects the shape of the structure included in the second decoded data using information indicating a configuration of the structure.

7. The image analyzing device according to claim 1 , wherein the measurement unit measures the position of the object to be inspected by calculating a distance from the reference line to the position of the object to be inspected using coordinates generated based on the reference line.

8. 3. The image analysis device according to claim 2, wherein when the plurality of objects to be inspected are attached to the structure at different times, the identification unit, the specification unit, and the measurement unit are executed each time an object is attached to the structure.

9. When the plurality of objects to be inspected are attached to the structure at different times, the identification unit, the specification unit, and the measurement unit are executed after all of the plurality of objects have been attached to the structure. The image analysis device according to claim 2 .

10. 2. The image analyzing device according to claim 1, wherein when a plurality of pieces of the second decoded data are acquired, the measurement unit outputs a single measurement result based on a plurality of measurement results for the position of the object to be inspected measured in each of the second decoded data.

11. The image analysis device according to claim 1 , further comprising a generating unit that generates the compression ratio map.

12. 12. The image analyzing device according to claim 11, wherein the generating unit generates a compression rate map in which the object region and the reference region are compressed to a second compression rate that is smaller than the first compression rate, and the region other than the object region and the reference region is compressed to a third compression rate that is larger than the first compression rate.

13. 12. The image analyzing device according to claim 11, wherein the generating unit generates a compression rate map in which the compression rate for all regions is changed to a third compression rate that is greater than the first compression rate, at least after the measuring unit has completed measuring the position of the object to be inspected.

14. 14. The image analyzing device according to claim 13, wherein the generating unit generates a compression rate map in which all regions are changed to the third compression rate at least until a structure before the object to be inspected is included in the first image data.

15. The identification unit has: a first identification unit that identifies a region of an object to be inspected by performing image recognition processing on the first decoded data; a second identification unit that performs image recognition processing on the second decoded data to individually identify regions of the object to be inspected; The image analysis device according to claim 4 , further comprising:

16. the identification unit identifies the object region by calculating a difference in pixel value between the first decoded data including the structure before the object to be inspected is attached and the first decoded data including the structure after the object to be inspected is attached; The image analyzing device according to claim 1 , wherein the identifying unit identifies the reference region using the identified object region and information indicating a configuration of the structure.

17. decoding the first image data encoded at the first compression rate, and when first decoded data is acquired, analyzing the first decoded data to identify regions for specifying an object region including an object to be inspected and a reference region including a reference line; The object area and the reference area are identified using the identified area and information indicating the configuration of a structure to which the object to be inspected is attached. a compression rate map is generated in which the object region and the reference region are compressed at a second compression rate that is smaller than the first compression rate, and second decoded data is obtained by decoding second image data that has been encoded using the compression rate map; and the position of the object to be inspected based on the reference line is measured by analyzing the second decoded data. An image analysis method in which processing is performed by a computer.

18. decoding the first image data encoded at the first compression rate, and when first decoded data is acquired, analyzing the first decoded data to identify regions for specifying an object region including an object to be inspected and a reference region including a reference line; The object area and the reference area are identified using the identified area and information indicating the configuration of a structure to which the object to be inspected is attached. a compression rate map is generated in which the object region and the reference region are compressed at a second compression rate that is smaller than the first compression rate, and second decoded data is obtained by decoding second image data that has been encoded using the compression rate map; and the position of the object to be inspected based on the reference line is measured by analyzing the second decoded data. An image analysis program that allows a computer to carry out the processing.

Images