Estimation Support Device, Estimation Support Method, and Program

Abstract

The present invention acquires image data obtained by imaging an object that includes a plurality of parts and determines the degree of acceptability for each part that the object includes on the basis of the acquired image data. Meanwhile, the present invention calculates an estimate of costs required for the repair of the object on the basis of the degree of acceptability for each part determined here and outputs the estimate.

Description

Technical Field

[0001] The present invention relates to an estimate support device, an estimate support method, and a program.

Background Art

[0002] When performing repairs on vehicles such as automobiles, for example, sheet metal repairs, it is common to create an estimate of the repair cost in advance. This estimate is made by a skilled person with repair skills inspecting the vehicle to identify deformed areas and areas where problems have occurred, and then considering the difficulty of repairing the deformed parts and the availability of replacement parts if they are to be replaced. This is the current situation.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Such a conventional manual estimate creation requires specialized knowledge and time, and furthermore, even if an estimate is created, the repair is not necessarily carried out. Therefore, it only incurs costs and does not benefit either the repair requester or the side receiving the repair request.

[0005] In Patent Document 1, as one means for detecting cracks or the like in metal parts, a means for processing a microscopic image is provided to predict the repair cost from the degree of deterioration or damage. Although inspections can be performed on each part, the quality of each part has not been determined from an image of the overall appearance of an automobile or the like to be repaired. Also, cost increases in cases where, for example, some parts have been replaced with non-standard parts cannot be considered.

[0006] The present invention has been made in view of the above circumstances, and one of its objects is to provide an estimate support device, an estimate support method, and a program that can support the estimation of the cost required for repairing an object including a plurality of parts.

Means for Solving the Problems

[0007] One aspect of the present invention for solving the problems of the above conventional example is an estimate support device, including: an acquisition means for acquiring image data obtained by imaging an object including a plurality of parts; a determination means for determining the quality level of each part included in the object based on the acquired image data; and a calculation means for calculating and outputting an estimate of the cost required for repairing the object based on the quality level of each part determined by the determination means.

Effects of the Invention

[0008] According to the present invention, it is possible to support the estimation of the cost required for repairing an object including a plurality of parts.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0010] Embodiments of the present invention will be described with reference to the drawings. As illustrated in FIG. 1, the estimation support device 1 according to the embodiments of the present invention is a computer device connected to an imaging device 2, and includes a control unit 11, a storage unit 12, an operation unit 13, a display unit 14, and an input / output unit 15.

[0011] Here, the control unit 11 is a program control device such as a CPU, and operates according to a program stored in the storage unit 12.

[0012] In the example of this embodiment, the control unit 11 acquires image data obtained by imaging an object such as an automobile including a plurality of parts, and based on the acquired image data, determines the quality level for each part included in the object. Then, the control unit 11 calculates and outputs an estimate of the cost required for repairing the object based on the quality level for each part determined here. The detailed operation of this control unit 11 will be described later.

[0013] The storage unit 12 includes at least one memory device such as a RAM, and holds a program executed by the control unit 11. This program may be stored and provided in a computer-readable and non-transitory recording medium, and stored in this storage unit 12. Further, this storage unit 12 also operates as a work memory of the control unit 11.

[0014] The operation unit 13 is a mouse, a keyboard, etc., accepts a user's operation, and outputs information representing the operation to the control unit 11. The display unit 14 is a display, etc., and displays information according to an instruction input from the control unit 11.

[0015] The input / output unit 15 is a USB (Universal Serial Bus) interface or the like, and outputs information to peripheral devices such as a printer according to an instruction input from the control unit 11. In this embodiment, an imaging device 2 is connected to the input / output unit 15, and the information input from the imaging device 2 is output to the control unit 11.

[0016] The imaging device 2 includes a camera C having an imaging angle capable of imaging the entire object, and a controller R that holds the image data (which may be moving image data) captured by the camera and outputs it to the estimate support device 1. In the example of this embodiment, the imaging device 2 may be a general digital camera, and the user who intends to create an estimate operates this imaging device 2 to capture at least one image data of the object. The shooting may be performed from a plurality of different viewpoints, and image data (which may be moving image data) obtained by imaging the object from various angles may be obtained. When the imaging device 2 is connected to the estimate support device 1, the image data obtained by the imaging is output to the estimate support device 1.

[0017] In an example of this embodiment, the imaging device 2 may obtain image data of the object imaged from a plurality of directions and output it to the estimate support device 1. For example, when the object is a vehicle, the imaging device 2 images the front, left side, right side, and back of the vehicle, and outputs the image data obtained by the imaging to the estimate support device 1.

[0018] Next, the operation of the control unit 11 of the estimate support device 1 will be described. In the example of this embodiment, as illustrated in FIG. 2, the control unit 11 functionally includes an image reception unit 21, a quality determination unit 22, and a calculation unit 23. The image reception unit 21 receives the image data of the object captured by the imaging device 2 connected to the input / output unit 15 and holds it in the storage unit 12.

[0019] The pass / fail determination unit 22 refers to the image data received by the image reception unit 21 (each of the moving image data, etc., and each of the plurality of still image when there are multiple still images), and determines the pass / fail degree for each part of the object imaged in the image data.

[0020] In an example of this embodiment, the pass / fail determination unit 22 includes a part recognition unit 221 and a pass / fail information acquisition unit 222.

[0021] [Operation of the part recognition unit] The part recognition unit 221 performs processes such as contour line extraction and region division by the contour line, and specifies each part where each component (part) of the object is imaged. Note that the method for specifying the imaged part of the part here is not limited to the method of dividing by the contour line.

[0022] For example, the part recognition unit 221 recognizes the posture of the object in the front-rear direction, left-right direction, etc., and compares it with the three-dimensional model data (here, data representing the outer shape of the part by a set of polygons, etc., and as an example, data in the OBJ format corresponds) prepared in advance for each part and the recognized posture of the object, and may estimate the imaged part of the part by comparing it with the virtual object image obtained by arranging the three-dimensional data of the part in the virtual three-dimensional space.

[0023] In this case, the part recognition unit 221 enlarges / shrinks the three-dimensional model data of each part included in the object according to the image of the imaged object, and also adjusts the rotation angle and position in the three-dimensional space, arranges and renders them, and reproduces the image data of the imaged object. Then, the part recognition unit 221 estimates the imaged part of the part by extracting the part where the three-dimensional model data of each part is imaged from this rendering result.

[0024] Also, both the region division by the contour line and the method using the three-dimensional model data may be combined (for example, if it is recognized as a part by any method, it is treated as a part), and the imaged part of each part of the object may be recognized.

[0025] [Another Example of the Operation of the Part Recognition Unit] Also, the part recognition unit 221 may be processed as follows without using three-dimensional model data. Another operation example of the part recognition unit 221 according to the present embodiment is as follows. That is, the part recognition unit 221 according to this another example includes a background removal unit 2211, a contour enhancement unit 2212, a part region determination unit 2213, and an image extraction unit 2214, as illustrated in FIG. 3.

[0026] The background removal unit 2211 removes a portion other than the portion where the object is photographed from the image data input from the imaging device 1 (referred to as processing target image data) as a background portion, and removes the background portion from the image data to extract a foreground portion. This process may be performed by a general process for extracting a foreground. As an example, the background removal unit 2211 can be realized by a segmentation process such as Grab Cut.

[0027] Also, the background removal unit 2211 in this example of the present embodiment may be capable of being processed interactively, such as Grab Cut. In Grab Cut, a mask is set, and since the foreground region is extracted using the set mask, the foreground region extracted can be adjusted according to the content of the mask setting.

[0028] Therefore, the background removal unit 2211 initially performs foreground extraction by Grab Cut processing using a predetermined mask, and displays the result to show it to the user. Then, the background removal unit 2211 accepts the designation of the mis-extracted location from the user. The user designates the portion that was extracted as a foreground although it is a background and the portion that was extracted as a background although it is a foreground, respectively, and sets a mask.

[0029] The background removal unit 2211 uses the mask set by the user to perform the grab cut process again, extracts the foreground, and displays the result to show it to the user. When the user determines that the foreground has been extracted based on the result, the user instructs the background removal unit 2211 to complete the process. Also, when the user determines that there is an error in the foreground extraction, the user re-sets the mask, and the background removal unit 2211 repeatedly performs the grab cut process using the mask set by the user.

[0030] When the background removal unit 2211 receives an instruction to complete the process from the user, it outputs the image data of the foreground extraction result shown to the user at that time to the contour enhancement unit 2212.

[0031] The contour enhancement unit 2212 performs a process of enhancing the contour portion on the image data after background removal. Specifically, this contour enhancement unit 2212 executes the contour enhancement process by so-called unsharp masking. However, the contour enhancement process is not limited to unsharp masking. For example, after performing a contour line extraction process such as canny, the pixel values of the pixels on the extracted contour line can be used to overwrite the pixel values of the pixels adjacent to the pixels on the contour line, and the contour portion can be enhanced by such a process.

[0032] The part region determination unit 2213 performs segmentation on the foreground image data whose contour has been enhanced by the contour enhancement unit 2212 by a method such as the widely known watershed. By this process, the regions separated by the contour lines are divided, and the regions of each part are extracted. The part region determination unit 2213 sets a unique label for each of the extracted part regions for the pixels within the region, and performs a labeling process for each part.

[0033] The image extraction unit 2214 extracts the pixel group corresponding to each pixel group with a common label from the process target image data for each pixel group with a common label, and generates image data for each part.

[0034] [Another example of contour extraction] Also, the above-mentioned contour enhancement unit 2212 may extract the contour by the following process. That is, in one example of the present embodiment, since the object is generally the body of an automobile made of metal with various color variations, in the object imaged by the imaging device 2, the surrounding scenery of the object may be captured, or the luminance change at the boundary may vary depending on the color tone of the object. In such a case, the contour line of one part may not be extracted as a continuous edge.

[0035] Therefore, the contour enhancement unit 2212 grayscales the image data after background removal to extract only luminance information, and applies a differential filter such as a Prewitt filter to the grayscaled image data to search for the minimum points in the respective axial directions of the horizontal direction (X-axis direction) and the vertical direction (Y-axis direction) of each pixel constituting the image data (a pixel whose pixel value is smaller than that of the surrounding pixels by more than a predetermined ratio).

[0036] FIG. 4 shows an example of grayscaling the image data in which the rear end portion of an automobile is imaged (FIG. 4(a)). In the example of FIG. 4(a), since there is a light source on the right side of the vehicle body, the rear of the vehicle body is slightly dark, and it is illustrated as an example in which the side surface of an adjacent vehicle is captured.

[0037] The contour enhancement unit 2212 scans each pixel constituting the image data along one-dimensional lines in the respective axial directions of the horizontal direction (X-axis direction) and the vertical direction (Y-axis direction) to search for the minimum points. For example, when a Prewitt filter is applied to each pixel along the line segment Q in the Y-axis direction to search for a portion where the luminance changes rapidly (the amount of change is larger than a predetermined value and is the pixel with the minimum value), pixels that become the minimum points are detected at the boundary of the part where the luminance changes rapidly (P), but pixels that become the minimum points are not detected in a portion where the luminance change is gentle such as a reflection (R). Therefore, the boundary of the part is mainly detected (FIG. 4(b)).

[0038] The contour enhancement unit 2212 further executes a process of removing isolated points from the obtained image data. That is, the contour enhancement unit 2212 selects one of the detected local minimum points (a local minimum point that has not been selected in the past) and issues a unique identifier specific to the selected local minimum point. The contour enhancement unit 2212 records the issued unique identifier in association with information identifying the selected local minimum point. Then, the contour enhancement unit 2212 refers to the 8 pixels (8-neighborhood) in the vicinity of the selected local minimum point. If there is another (unselected) local minimum point different from the selected local minimum point in the 8-neighborhood, the information identifying the other local minimum point is also recorded in association with the information identifying the selected local minimum point and the issued unique identifier specific to the selected local minimum point. Then, the contour enhancement unit 2212 selects the other local minimum point (sequentially for each if there are multiple) and continues the process of referring to its 8-neighborhood.

[0039] Also, if there is no unselected local minimum point in the 8-neighborhood of the selected local minimum point, the contour enhancement unit 2212 searches for whether there is any other unselected local minimum point. If there is an unselected local minimum point, the contour enhancement unit 2212 selects the local minimum point, issues a unique identifier, and repeats the above process. Further, when there are no unselected local minimum points left, the contour enhancement unit 2212 ends the process.

[0040] After performing the above processes, the contour enhancement unit 2212 finds an identifier associated with a number of local minimum points less than a predetermined number, and deletes the local minimum points associated with the found identifier from the record (isolated point removal process). Thereby, isolated points or a set of minute local minimum points are removed.

[0041] The contour emphasizing unit 2212 refers to the record after the isolated point removal process, and outputs, as a contour line, a line segment formed by connecting the minimum points associated with each identifier (Fig. 4(c)). Thus, in an example of the part recognition unit 221 of the present embodiment, the image data is converted to grayscale, the minimum points are searched from each line in the axial directions of the horizontal and vertical directions respectively, and a group of minimum points that are adjacent to each other among the searched minimum points is found, and a process of detecting a continuous minimum point group is executed. Among the detected minimum point groups, a line segment constituted by the minimum point groups excluding the minimum point groups in which the number of minimum points belonging to the minimum point groups is less than a predetermined number is determined as the contour line of the part, and partial image data for each part (for each imaged part of the part) is obtained using the contour line.

[0042] [Identification of the writing boundary] Furthermore, for each of the contour lines of the parts determined by the above-described process, the contour emphasizing unit 2212 extracts the region including the contour line from the image after grayscale conversion, and obtains the second-order differential value (the difference of the differences of adjacent pixels) of the pixels in the region. Specifically, the contour emphasizing unit 2212 sequentially selects one by one the contour lines of the parts determined by the above-described process. Then, the image after grayscale conversion is rotationally transformed so that the selected contour line becomes a line extending in a predetermined axial direction, for example, the vertical direction of the image data. This is performed, for example, by rotating the contour line so that the horizontal size of the rectangle circumscribing the contour line becomes the smallest when the rectangle is detected.

[0043] Then, the contour emphasizing unit 2212 calculates the difference of the differences of the pixel values of adjacent pixels in the direction perpendicular to the above-described predetermined axial direction (the pixel scanning direction). Specifically, when pixels with pixel values (luminances) of P1, P2, P3, P4,... are arranged from left to right (the pixel scanning direction), (P2 - P1) - (P3 - P2), which is the difference of the differences, is obtained as the second-order differential value.

[0044] When the second derivative value of this exceeds a predetermined second derivative threshold, the contour emphasizing unit 2212 determines that the contour line is included in the pixel related to the second derivative value (in the above example, the pixel with pixel values P1, P2, and P3). If the second derivative value does not exceed the predetermined second derivative threshold, it is determined that the contour line is not included in the pixel related to the second derivative value.

[0045] According to this process, it is possible to prevent the contour of the image reflected on the vehicle body or the like from being erroneously determined as the contour line of the part. That is, the luminance of the contour line due to reflection is dispersed by the material such as the body surface. As illustrated in FIG. 7(a), when scanning the pixels across the reflected contour line, the luminance of the pixels changes smoothly.

[0046] On the other hand, for the contour line that is the boundary of the actual part of the body, when scanning the pixels across the contour line, the luminance of the pixels changes abruptly (FIG. 7(b)). Therefore, in order to distinguish these, in this embodiment, the second derivative (absolute value) of the luminance is used. The second derivative result corresponding to the example of FIG. 7(a) is shown in FIG. 7(c), and the second derivative result corresponding to the example of FIG. 7(b) is shown in FIG. 7(d). As can be seen in FIGS. 7(c) and 7(d), for the contour line representing the boundary of the non-reflected part, the second derivative value at the contour line is a relatively large value, while for the reflected contour line, the second derivative value at the contour line is a sufficiently small value. Therefore, a threshold value (second derivative threshold value) for distinguishing these values is determined experimentally and empirically and used.

[0047] In this way, for the contour line in the image portion determined not to include the contour line by the method using the second derivative value described here among the contour lines of the parts determined by the process described above, the contour emphasizing unit 2212 may not treat it as the contour line of the part in the following process.

[0048] That is, the contour enhancement unit 2212 determines, among the contour lines of the parts determined by the above-described processing, the contour lines that are judged to be included in the image portion where the contour lines are judged to be included by the method using the second derivative value described here (the contour lines where the pixel values change abruptly across the contour lines) as the contour lines representing the actual part boundaries rather than the reflected ones. Then, the part region determination unit 2213 performs segmentation on the foreground image data surrounded by the contour lines judged to be contour lines here by a widely known method. By this processing, the regions separated by the contour lines are divided, and the regions of each part are extracted.

[0049] The pass / fail information acquisition unit 222 generates information (proposed repair information) necessary for determining the required repair for each part. In an example of the present embodiment, the pass / fail information acquisition unit 222 generates an estimated image in a non-damaged state for each imaged portion of the part based on the received image data. This estimated image can be obtained by correcting the pixel values of the image portions of the damage, dent, etc. in the imaged part with reference to the pixel values of the surrounding image portions when there are damage, dents (dents), etc. in the imaged part (for example, the same processing as the processing of the repair tool in Adobe Photoshop (registered trademark) of the United States may be executed).

[0050] Specifically, the pass / fail information acquisition unit 222 extracts the images of the portions of each part recognized by the part recognition unit 221 from the received image data, and generates partial image data for each part. This partial image data is rectangular image data circumscribing the image of the portion of the extracted part, enclosing the image of the portion of the part, and setting the pixels other than the portion of the part as invalid pixels (for example, transparent pixels).

[0051] The pass / fail information acquisition unit 222 generates, for the partial image data of each part, a binarized image obtained by further binarizing the partial image data and an image (hue component image) obtained by extracting only the hue component of the partial image data. Here, for binarization, widely known processes such as a process of dividing the image into blocks, binarizing with a threshold value weighted by a Gaussian distribution for each block, or a binarization process using Otsu's method can be adopted. When pixels other than the part portion are included in a block, the pass / fail information acquisition unit 222 ignores the pixel values of the pixels and treats them as having no pixel values.

[0052] The pass / fail information acquisition unit 222 performs blob analysis on the binarized image and the hue component image generated for each partial image data of each part, and identifies parts (hereinafter referred to as parts to be repaired) where scratches, dents, etc. have occurred for each part.

[0053] For the partial image data in which parts to be repaired are identified among the partial image data of each part, the pass / fail information acquisition unit 222 replaces the pixel value of each pixel included in the part to be repaired with the pixel value of a pixel in the part portion that is the closest to the pixel and is outside the part to be repaired (repair process). Alternatively, in this repair process, the pass / fail information acquisition unit 222 replaces the pixel value of each pixel included in the part to be repaired with the average value of the pixel values of a plurality of pixels in the part portion that are close to the pixel and are outside the part to be repaired.

[0054] Furthermore, as a repair process, the pass / fail information acquisition unit 222 may obtain texture information from an array of a plurality of pixels in the part portion for pixels outside the part to be repaired and fill the pixels in the part to be repaired with the texture.

[0055] Through this repair process, the pass / fail information acquisition unit 222 generates, for each partial image data of each part, an estimated image in a state where the part is intact, corresponding to the partial image data.

[0056] The pass / fail information acquisition unit 222 compares, for each partial image data of each part, the estimated image in a state where the part is intact with the partial image data (original data) of each part before the restoration process. The pass / fail information acquisition unit 222 performs this comparison by means of binary comparison in which the estimated image and the original data are each binarized and compared, and color comparison in which images obtained by extracting the hue components from the estimated image and the original data are compared.

[0057] Based on the result of the comparison, the pass / fail information acquisition unit 222 obtains information regarding the degree of deformation of the part, such as the size, direction, width of the part to be restored, the type of damage included in the part to be restored, or the depth of the damage or dent included in the part to be restored (obtained from the difference in hue from surrounding pixels, etc.).

[0058] Then, the pass / fail information acquisition unit 222 outputs, as proposed repair information, the information indicating the size, direction, width of the part to be restored for each part obtained here, the type of damage included in the part to be restored, and the depth of the damage and dent included in the part to be restored.

[0059] Alternatively, the pass / fail information acquisition unit 222 may generate proposed repair information while classifying the types of damage included in the part to be restored by the following method.

[0060] [Extraction of linear scratches] The pass / fail information acquisition unit 222 extracts, for example, the linear scratch (referred to as a linear scratch) portion from the partial image data of each part by the following method. For each partial image data of each part, the pass / fail information acquisition unit 222 converts the partial image data (image data in the RGB color space) into image data in the HSV (Hue, Saturation, Value) color space. Then, the pass / fail information acquisition unit 222 compares the threshold value (kmeans threshold) K when classifying the brightness (V) of the pixel value in the converted image data by the kmeans method with the average value (mean) M of the brightness (V) of the pixel value, and determines that the color tone of the scratch is close to black when the average value M is greater than the kmeans threshold K (M > K), and determines that the color tone of the scratch is close to white when M > K is not satisfied.

[0061] When the quality information acquisition unit 222 determines that the color tone of the scratch is close to white, it extracts the image of the color component of the S channel (chroma space) and the image of the color component of the V channel (brightness space) from the image data converted into the HSV color space, generates binarized images obtained by binarizing each of them, and synthesizes (adds) the two generated binarized image data to generate labeling target data. Note that the binarization threshold may be the kmeans threshold. However, for the image data of the V channel, with the kmeans threshold as the initial value, while presenting the temporary binarization result to the user, the binarization threshold is changed according to the user's instruction, and binarization may be performed using the threshold when the user determines that the binarized image in which the scratch part is extracted is appropriate.

[0062] In addition, when the quality information acquisition unit 222 determines that the color tone of the scratch is close to black, it extracts the image of the color component of the V channel (brightness space) from the image data converted into the HSV color space, and uses the extracted image of the color component of the V channel as the labeling target data as it is.

[0063] The quality information acquisition unit 222 performs a labeling process (a process of setting different labels for each location where pixels with a distinguishable color tone from the background pixels are continuous) on the labeling target data obtained by any of these processes. This process can be executed, for example, by applying a process such as connectComponents in opencv.

[0064] The quality information acquisition unit 222 sequentially selects rectangular regions circumscribing the pixel blocks found in the labeling process and obtains the areas of the selected rectangular regions. When the obtained area is "0" or exceeds a threshold (area threshold) determined by a predetermined method, the quality information acquisition unit 222 skips the following processing for the rectangular region and selects the next rectangular region (if there is no rectangular region to be selected, that is, if the processing for all rectangular regions has been completed, the process of extracting linear scratches from the partial image data that is currently the processing target is terminated).

[0065] Note that the area threshold here may be a predetermined value, or may be a value obtained by multiplying the area of the partial image data of the part being processed by a predetermined ratio (a ratio less than 1).

[0066] When the obtained area is greater than "0" and less than the area threshold determined by the above-described predetermined method, the pass / fail information acquisition unit 222 executes the following process. That is, the pass / fail information acquisition unit 222 obtains the aspect ratio (the value obtained by dividing the length of the long side by the length of the short side) of the selected rectangular region, and determines whether this aspect ratio is smaller than a predetermined aspect ratio threshold (not an elongated shape), or whether the value obtained by dividing the area of the pixel block found by the labeling process (the number of labeled pixels) by the area of the rectangular region (the rectangle circumscribing the pixel block) exceeds a predetermined area ratio threshold (in the case of a large scratch). In such a case, the following process is skipped and the next rectangular region is selected (if there is no rectangular region to be selected, that is, if the processing for all rectangular regions has been completed, the process of extracting linear scratches from the partial image data currently being processed is terminated).

[0067] When the aspect ratio is greater than or equal to a predetermined aspect ratio threshold (is an elongated shape), or when the value obtained by dividing the area of the pixel block found by the labeling process (the number of labeled pixels) by the area of the rectangular region (the rectangle circumscribing the pixel block) is less than a predetermined area ratio threshold (in the case of a relatively small scratch), the pass / fail information acquisition unit 222 determines that there is a linear scratch in the image portion represented by the pixel block found by the labeling process within the selected rectangular region, and associates information indicating that the type of the scratch is "linear scratch", the area (size) of the rectangular region (or pixel block), and information for specifying the position within the part corresponding to the partial image data (information obtained from the coordinate information of the rectangular region) with the information for specifying the part corresponding to the partial image data being processed, includes the information in the proposed repair information, and repeats the process of selecting the next rectangular region. Note that when there is no rectangular region to be selected, that is, when the processing for all rectangular regions has been completed, the pass / fail information acquisition unit 222 terminates the process of extracting linear scratches from the partial image data currently being processed.

[0068] [Extraction of scratches] The pass / fail information acquisition unit 222 may also perform a process of extracting scratches following or before the process of extracting these linear scratches.

[0069] In this process, for each partial image data of each part, the pass / fail information acquisition unit 222 obtains a threshold value (kmeans threshold value) when classifying the values of the V channel (brightness component) of the pixel values of the partial image data (image data in the HSV color space) by the kmeans method. Then, the pass / fail information acquisition unit 222 determines a range of a predetermined value including this kmeans threshold value as a threshold range. For example, the pass / fail information acquisition unit 222 sets the value obtained by subtracting a predetermined value t from the kmeans threshold value Tk, i.e., Tk - t, as the lower limit, and the value obtained by adding the predetermined value t to the kmeans threshold value Tk, i.e., Tk + t, as the upper limit (however, when the obtained lower limit is less than "0" (the minimum value of brightness V), the lower limit is set to "0", and when the obtained upper limit exceeds the maximum value that brightness V can take, e.g., "255", the upper limit is set to the maximum value) to determine the threshold range.

[0070] Then, the pass / fail information acquisition unit 222 uses the value of the lower limit of this threshold range as the initial value, and uses a plurality of values up to the value of the upper limit of the threshold range as thresholds to generate an image obtained by binarizing the partial image data using the thresholds, and synthesizes the binarized images (accumulates the binarized pixel values to generate a multi-valued image).

[0071] The pass / fail information acquisition unit 222 performs a labeling process on the multi-valued image obtained by the synthesis, sequentially selects rectangular regions circumscribing the pixel blocks found in the labeling process, and obtains the areas of the selected rectangular regions. When the obtained area is "0", the pass / fail information acquisition unit 222 skips the following process for the rectangular region and selects the next rectangular region (if there are no rectangular regions to be selected, that is, if the processing for all rectangular regions has been completed, the process of extracting scratches from the partial image data currently being processed is terminated).

[0072] When the obtained area is greater than "0", the defect determination information acquisition unit 222 calculates the aspect ratio of the selected rectangular area (the value obtained by dividing the length of the long side by the length of the short side), and determines whether this aspect ratio is greater than a predetermined aspect ratio threshold (i.e., the shape is elongated), or whether the value obtained by dividing the area of the pixel block found in the labeling process (the number of labeled pixels) by the area of the rectangular area (the rectangle circumscribing the pixel block) exceeds a predetermined area ratio threshold (i.e., it is a large defect). If either of these conditions is met, the following processing is skipped and the next rectangular area is selected. (If there are no more rectangular areas to select, i.e., if the processing for all rectangular areas has been completed, the process of extracting scratches from the partial image data currently being processed ends.) Note that the aspect ratio threshold and the area ratio threshold here may be the same as or different from the respective thresholds used in the linear scratch extraction process.

[0073] When the defect determination information acquisition unit 222 finds a pixel block that is determined to be an image of a scratch that is not elongated and not a large defect, it records the label of the pixel block as the processing target label. Then, the defect determination information acquisition unit 222 repeats the process of selecting the next rectangular area. If there are no more rectangular areas to select, i.e., if the processing for all rectangular areas has been completed, the defect determination information acquisition unit 222 generates a polygonal area that surrounds the pixel block corresponding to the label recorded as the processing target label. The generation of this polygonal area may be a process of generating a so-called convex hull, or a process of generating an alpha shape (a shape formed by a simple and piecewise linear group of curves associated with the shape of a finite set of points on a plane) (since both are widely known, detailed descriptions are omitted here).

[0074] The defect determination information acquisition unit 222 associates information indicating that the type of defect is "scratch", the area of the polygonal area obtained here, and information specifying the position of the polygonal area within the part corresponding to the partial image data being processed with the information specifying the part corresponding to the partial image data being processed, includes it in the proposed repair information, and repeats the process of selecting the next rectangular area.

[0075] [Estimation of Depth of Scratches and Dents] The pass / fail information acquisition unit 222 may also perform a process of extracting the depth of scratches and dents following the process of extracting these linear scratches and abrasions.

[0076] The deeper the scratches and dents are, the lower the brightness at their central parts. Therefore, the pass / fail information acquisition unit 222 obtains the brightness difference between the central part and the peripheral part of the scratches and dents as follows. That is, for each partial image data of each part, the pass / fail information acquisition unit 222 obtains a threshold value (kmeans threshold value) when classifying the values of the V channel (brightness component) of the pixel values of the partial image data (image data in the HSV color space) by the kmeans method. Then, the pass / fail information acquisition unit 222 determines a range of predetermined values including this kmeans threshold value as a threshold range. For example, the pass / fail information acquisition unit 222 sets the lower limit as the value obtained by subtracting a predetermined value τ (for example, this τ may be the absolute value of the difference between the kmeans threshold value Tk and the average M of the pixel values of the V channel) from the kmeans threshold value Tk, i.e., Tk - τ, and sets the upper limit as the value obtained by adding the predetermined value τ to the kmeans threshold value Tk, i.e., Tk + τ (however, when the obtained lower limit is less than "0" (the minimum value of brightness V), the lower limit is set to "0", and when the obtained upper limit exceeds the maximum value that brightness V can take, for example, "255", the upper limit is set to the maximum value) to determine the threshold range.

[0077] Then, the pass / fail information acquisition unit 222 uses the value of the lower limit of this threshold range as the initial value, and uses a plurality of values up to the value of the upper limit of the threshold range as thresholds to generate images obtained by binarizing the partial image data using each threshold, and extracts a closed curve (contour) of equal brightness from each of the plurality of binarized images generated using each threshold.

[0078] The pass / fail information acquisition unit 222 selects an isochromatic closed curve that is obtained from the image binarized with a higher brightness threshold among the extracted isochromatic closed curves, does not include the pixel block determined to be a linear scratch or a rubbed scratch within the closed curve, and encloses the isochromatic closed curve obtained from the image binarized with a relatively lower brightness threshold (when it is an isochromatic closed curve obtained from the image with the lower threshold value, it does not include the pixel block determined to be a linear scratch or a rubbed scratch within the closed curve). Then, the pass / fail information acquisition unit 222 obtains the difference between the brightness value of the pixel with the lowest brightness in the region corresponding to the selected isochromatic closed curve in the V channel (brightness component) image of the partial image data and the average value of the pixels outside the region corresponding to the selected isochromatic closed curve and adjacent to the selected isochromatic closed curve, and estimates the depth of the concave portion such as a scratch or a dent based on this difference. Specifically, assuming that this difference represents the value of the depth of the concave portion, information indicating that the type is "dent", information specifying the position of the selected isochromatic closed curve within the part corresponding to the partial image data to be processed, and the value of the estimated depth of the concave portion are associated and included in the proposed repair information, and the process is repeated from the process of selecting the following rectangular region.

[0079] The pass / fail information acquisition unit 222 repeatedly executes the process of generating the proposed repair information including the above-described processes for the partial image data of each part.

[0080] Then, the pass / fail information acquisition unit 222 outputs the proposed repair information (information indicating the size, direction, width of the repair target portion for each part, the type of scratch included in the repair target portion, and the depth of the scratch or dent included in the repair target portion) obtained here to the calculation unit 23.

[0081] The calculation unit 23 calculates an estimate of the cost required for repairing the object while referring to the proposed repair information for each part output by the pass / fail determination unit 22. Specifically, this calculation unit 23 calculates the cost corresponding to the degree of necessity of repair for each part. For example, when the degree of deformation, which is the degree of necessity of repair, is represented by the area of the dent (the size of the part to be repaired), the calculation unit 23 presets unit price information representing the amount of cost per unit area of the dent, and multiplies the unit price information by the area represented by the above information, which is the degree of necessity of repair, to calculate the estimated cost.

[0082] [Operation] This embodiment has the above configuration and operates as follows. The user of the estimation support device 1 of this embodiment uses the imaging device 2 to capture an image of a vehicle that is the object of repair estimation. Then, the image data of the object is input to the estimation support device 1 (Fig. 5: S1).

[0083] The estimation support device 1 refers to the received image data (each of them if there are multiple) and determines the pass / fail degree for each part of the object imaged in the image data. That is, the estimation support device 1 specifies the image portion (pixel range) of each part of the vehicle imaged in the received image data from the image of the vehicle (S2). This process can be realized as a process such as dividing the area by a contour line.

[0084] The estimation support device 1 extracts the image of each part specified in step S2 from the received image data and generates partial image data for each part. As described above, this partial image data is rectangular image data circumscribing the image of the part extracted, enclosing the image of the part, and setting the pixels other than the part as invalid pixels (e.g., transparent pixels).

[0085] The quotation support device 1 repeatedly executes the following process for the partial image data of each part obtained in step S3. The quotation support device 1 selects one of the unselected partial image data of each part obtained in step S3 (S4), and executes a process of generating repair proposal information based on the selected partial image data (S5).

[0086] As an example, the generation of this repair proposal information is performed as follows. That is, as illustrated in FIG. 6, the quotation support device 1 generates a binarized image obtained by binarizing the partial image data selected in step S4, and an image (hue component image) obtained by extracting only the hue component of the partial image data (S11).

[0087] Then, the quotation support device 1 performs blob analysis on the binarized image and the hue component image generated in step S11, and identifies a portion where scratches, dents, etc. have occurred (hereinafter referred to as a repair target portion) (S12). If no scratches, dents, etc. are imaged in the image of the part included in the selected partial image data (if there are no scratches, etc. on the part or they are not visible as an appearance), the quotation support device 1 does not identify the repair target portion.

[0088] The quotation support device 1 determines whether or not the repair target portion has been identified in step S12 (S13). If the repair target portion has been identified (S13: Yes), for the selected partial image data, the pixel value of each pixel included in the repair target portion is replaced with the pixel value of the pixel of the part portion that is closest to the pixel and is outside the repair target portion, etc., to generate an estimated image in a state where the part is intact (S14: repair estimation process).

[0089] The estimate support device 1 generates an image obtained by binarizing the estimated image generated here and the partial image data (original data) before the restoration estimation process, and an image obtained by extracting the hue component respectively, compares the binarized images with each other and the images of the hue components with each other, and based on the result of the comparison, obtains information such as the size, direction, width of the portion to be restored, and the depth of scratches and dents included in the portion to be restored. Then, the estimate support device 1 generates, as repair proposal information, information such as the size, direction, width of the portion to be restored of the part imaged in the selected partial image data, and the depth of scratches and dents included in the portion to be restored (S15).

[0090] On the other hand, if the portion to be restored is not specified in step S13 (S13: No), the estimate support device 1 generates information indicating that the part imaged in the selected partial image data is intact.

[0091] When the estimate support device 1 generates this repair proposal information for the partial image data of the selected part, if there is unselected partial image data among the partial image data of the part obtained in step S3, it returns to step S4, selects one of the unselected partial image data, and continues the process (S6).

[0092] As a result, the estimate support device 1 obtains information corresponding to the area of the portion that needs to be repaired for each part. The estimate support device 1 presets and holds unit price information representing the amount of repair cost per unit area of the part, multiplies the area represented by the above information, which is the degree of necessity of repair, by the unit price information, calculates an estimate of the repair cost for each part (S7), and outputs the calculated estimate of the repair cost for each part or the accumulated estimate thereof and presents it to the user (S8).

[0093] In addition, instead of the repair proposal information generation process in step S5 illustrated in FIG. 6, the estimate support device 1 may perform the following process. That is, the estimate support device 1 may sequentially execute the above-described linear scratch extraction process, the scratch extraction process, and the process of estimating the depth of scratches or dents on the partial image data selected in step S4 to generate repair proposal information.

[0094] Also according to this example, it becomes possible to calculate the estimated cost of repair for each part in step S7 and present it to the user in step S8.

[0095] [Modification Example] In the description so far of this embodiment, the repair cost has been calculated based on the deformation of the outer shape. However, this embodiment is not limited to this. For example, using the part identifier and information representing the degree of necessity for repairing the part identified thereby as input, and using a neural network or the like that has been pre-trained in a state of machine learning with information representing the degree of damage to the interior (parts that require repair and do not appear as deformation of the outer shape) when the same degree of damage occurred to the same part in the past as teacher information, the internal damage may be estimated.

[0096] In this case, the estimation support device 1 may present to the user the repair cost corresponding to the estimated internal damage together with the repair cost corresponding to the deformation of the outer shape.

Description of Reference Numerals

[0097] 1 Estimation support device, 2 Imaging device, 11 Control unit, 12 Storage unit, 13 Operation unit, 14 Display unit, 15 Input / output unit, 21 Image reception unit, 22 Pass / fail determination unit, 23 Calculation unit, 221 Part recognition unit, 222 Pass / fail information acquisition unit.

Claims

1. An acquisition means for acquiring image data obtained by imaging an object including a plurality of parts; A determination means for determining the quality level of each part included in the object based on the acquired image data; An arithmetic means for calculating and outputting an estimate of the cost required for repairing the object based on the quality level of each part determined by the determination means; comprising The determination means converts the image data into grayscale, searches for minimum points from each line in the axial directions of the horizontal and vertical directions, finds a pair of adjacent minimum points among the found minimum points, and executes a process of detecting continuous minimum point groups, Among the detected minimum point groups, a line segment composed of the minimum point groups excluding the minimum point groups in which the number of minimum points belonging to the minimum point group is less than a predetermined number is determined as the contour line of the part, and partial image data for each part is obtained using the contour line, An estimate support device for determining the quality level of each part included in the object based on the obtained partial image data for each part.

2. The estimate support device according to claim 1, wherein the object is a vehicle, and the quality level of the part includes information on the degree of deformation of the part.

3. An estimate support method using a computer, comprising a step in which an acquisition means acquires image data obtained by imaging an object including a plurality of parts; a step in which a determination means determines the quality level of each part included in the object based on the acquired image data; a step in which an arithmetic means calculates and outputs an estimate of the cost required for repairing the object based on the quality level of each part determined by the determination means; including The determination means converts the image data into grayscale, searches for minimum points from each line in the axial directions of the horizontal and vertical directions, finds a pair of adjacent minimum points among the found minimum points, and executes a process of detecting continuous minimum point groups, Among the detected minimum point groups, a line segment composed of the minimum point groups excluding the minimum point groups in which the number of minimum points belonging to the minimum point group is less than a predetermined number is determined as the contour line of the part, and partial image data for each part is obtained using the contour line, An estimate support method for determining the quality level of each part included in the object based on the obtained partial image data for each part.

4. A computer An acquisition means for acquiring image data obtained by imaging an object including a plurality of parts; A determination means for determining the quality level for each part included in the object based on the acquired image data; An estimation means for estimating and outputting the cost required for repairing the object based on the quality level for each part determined by the determination means; Function as; When functioning as the determination means, cause the computer to: Convert the image data to grayscale, search for local minima from each line in each axial direction of the horizontal and vertical directions, find a pair of adjacent local minima among the found local minima, and detect a continuous local minimum group; Among the detected local minimum groups, determine a line segment composed of local minimum groups excluding local minimum groups in which the number of local minima belonging to the local minimum group is less than a predetermined number as the contour line of the part, and obtain partial image data for each part using the contour line; A program for causing the quality level for each part included in the object to be determined based on the obtained partial image data for each part.

Images