Metering box defect self-adaptive detection method and equipment based on multi-view vision and medium

Abstract

The invention discloses a metering box defect self-adaptive detection method and device based on multi-view vision and a medium. The method comprises the steps that a multi-view optimal collection image set of a to-be-detected metering box is manufactured; calling a metering box model with the same model as the to-be-tested metering box in the database; constructing pose mapping information of the metering box model based on the multi-view optimal acquisition image set; planning the grid interval density in each defect detection area in the metering box model, and determining the position of each grid line in the defect detection area in the metering box model based on the grid interval density; generating a visual angle projection area of each defect detection area in the metering box model based on each camera, and a mapping area of the visual angle projection area in a corresponding camera acquisition image in a multi-visual angle optimal acquisition image set of the to-be-detected metering box; and calculating a color number deviation coefficient between each visual angle projection area and the corresponding mapping area, and generating defect detection feedback information of the to-be-detected metering box based on the color number deviation coefficients. The defect detection accuracy is high.

Description

Technical Field

[0001] The present invention relates to the technical field of visual inspection, and in particular to a method, device and medium for adaptively detecting defects in a metering box based on multi-viewing. Background Art

[0002] Meter boxes are core monitoring equipment for critical infrastructure, and their defect detection has important technical, economic, and safety significance. Traditional meter box defect detection relies on manual inspection, which not only has low detection efficiency but also poor detection accuracy. However, with the continuous development of computer vision technology, people use computer vision to collect appearance images of meter boxes and use image recognition technology to identify defects based on the collected results. This method effectively improves the detection efficiency of meter boxes while also providing a certain degree of detection accuracy for meter box defects.

[0003] Chinese patent application publication number CN119648657A discloses a method and system for detecting meter boxes. The method includes: obtaining meter data, structural data, and image data of a sample meter box; annotating the image data based on the meter data and structural data to obtain an annotated dataset; performing denoising, color enhancement, and diversity enhancement on the image data to obtain an enhanced image set; obtaining a preset base model, freezing a portion of the base model's structure, customizing the output layer structure of the base model, and unfreezing the base model to obtain a transfer learning model; inputting the label information and the enhanced image set into the transfer learning model for training to obtain a trained meter box detection model; and inputting image data of the meter box to be tested into the meter box detection model to obtain detection results.

[0004] A Chinese patent application with publication number CN119164949A discloses a method and system for component defect identification based on a target detection algorithm; the method includes: collecting component image data, performing target identification on the collected component image data based on a target detection algorithm, and obtaining identified components; based on the identified components, querying the defect type corresponding to the identified single component or the specific component group composed of multiple identified components from a component defect type query table, and taking the defect type corresponding to the queried single component or the specific component group as a defect identification task to form a defect identification task table; determining a first defect identification model corresponding to the defect type based on each defect identification task in the defect identification task table, inputting the component image data into the determined first defect identification model to obtain a first identification result; counting all first identification results with defects, and outputting all defects.

[0005] However, the existing method of detecting meter box defects through computer vision technology still has shortcomings, which is mainly reflected in the collected images of computer vision. Due to the differences in lighting conditions in the collection environment, the corresponding collected images often have color differences, and thus the recognition accuracy of color differences in the defect recognition results of the meter box is still relatively low. Summary of the Invention

[0006] In view of the deficiencies in the prior art, the present invention provides a method, device and medium for adaptively detecting defects in a metering box based on multi-vision.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for adaptively detecting defects in a meter box based on multi-viewing includes the following steps:

[0009] S1. Synchronously capture multi-view images of the meter box to be tested using multiple cameras, and construct a multi-view visual data analysis set for the meter box to be tested based on the preset positions of each camera; and select the optimal multi-view image set for the meter box to be tested from the multi-view visual data analysis set;

[0010] S2. Extract feature identifiers from each element in the multi-view optimally captured image set of the meter box to be tested using optical character recognition (OCR) technology, compare the feature identifier extraction results with a preset table in a database, obtain the model of the meter box to be tested in the preset table in the database, and call a meter box model with that model in the database; construct pose mapping information for the called meter box model based on the preset position of the camera to which each element in the multi-view optimally captured image set belongs and the corresponding image acquisition view angle vector;

[0011] S3. Querying the summary set of each defect detection area in the called meter box model and combining the abnormal defect detection results of the called meter box model in historical data, plan the grid spacing density within each defect detection area in the called meter box model, and determine the position of each grid line within the defect detection area in the called meter box model based on the grid spacing density;

[0012] S4. Obtain the viewing angle characteristics and illumination characteristics of the meter box model with normal detection results corresponding to the identified model, and the viewing angle characteristics and illumination characteristics constitute a relationship mapping data pair; use the viewing angle characteristics of the camera corresponding to each element in the multi-view optimal acquisition image set of the meter box to be tested to match the illumination characteristics of the meter box to be tested; and combine the posture mapping information of the called meter box model to generate the viewing angle projection area of ​​each defect detection area in the called meter box model based on each camera, as well as the mapping area of ​​the viewing angle projection area in the image captured by the corresponding camera in the multi-view optimal acquisition image set of the meter box to be tested;

[0013] S5. Obtain the unobstructed grid areas within the viewing angle projection area, as well as the positional relationship between each unobstructed grid area within the viewing angle projection area, and calculate the color number deviation coefficient between each viewing angle projection area and the corresponding mapping area, and generate defect detection feedback information of the meter box to be tested based on the color number deviation coefficient.

[0014] To optimize the above technical solutions, specific measures taken also include:

[0015] Furthermore, in S1, the multi-view visual data analysis set of the meter box to be tested includes multi-view images of the meter box to be tested that are synchronously acquired by multiple cameras at different time points, and the image of the meter box to be tested acquired by each camera is regarded as a view image of the meter box to be tested;

[0016] The specific process of selecting the optimal multi-view image set of the measuring box to be tested from the multi-view visual data analysis set of the measuring box to be tested is as follows:

[0017] The images of the meter box to be measured collected by the same camera at different time points are obtained respectively, and the pixel point of the center point of the meter box area to be measured in the image of the meter box to be measured is obtained, and recorded as the perspective reference point of the image of the meter box to be measured; the summary set of the perspective reference points of the images of the meter box to be measured collected by the same camera at different time points is recorded as the perspective reference point set of the corresponding camera; the element with the smallest sum of pixel distances from the remaining elements in the perspective reference point set of the corresponding camera is recorded as the perspective calibration point; any image of the meter box to be measured whose corresponding perspective reference point coincides with the perspective calibration point in the images of the meter box to be measured collected by the corresponding camera at different time points is used as the best perspective collected image of the meter box to be measured based on the corresponding camera, and the best perspective collected images of all cameras are combined into a multi-perspective best collection image set.

[0018] Furthermore, in S2, the specific process of extracting feature identification of each element in the multi-view optimally collected image set of the measuring box to be measured by using the OCR technology is as follows:

[0019] The image in the sliding window area is compared with the preset identifier respectively by using a sliding window method, and the feature identifier of the sliding window area whose similarity with the preset identifier is greater than a preset value is used as the corresponding feature identifier extraction result; the size of the sliding window is preset, and the similarity between the feature identifier of the sliding window area and the preset identifier is calculated by: proportionally enlarging the feature identifier extraction result of the sliding window area and making it equal to the length of the corresponding preset identifier, and the quotient of the number of overlapping pixels between the feature identifier extraction result of the sliding window area after the proportional enlargement and the preset identifier is divided by the number of pixels of the preset identifier as the similarity.

[0020] Furthermore, in S2, the feature identification extraction result is compared with the database preset table to obtain the model of the meter box to be tested in the database preset table:

[0021] When comparing the feature identification extraction results with the database preset table, the meter box model with the most overlapping feature identification extraction results is used as the model corresponding to the meter box to be tested;

[0022] The vector direction of the image acquisition perspective vector is the direction from the corresponding camera position to the center point of the corresponding perspective acquisition image, and the vector modulus is 1; the posture mapping information of the called meter box model includes the preset position of the camera belonging to each element in the multi-perspective optimal acquisition image set of the meter box to be measured and the corresponding image acquisition perspective vector.

[0023] Furthermore, in S3, the grid spacing density within each defect detection area in the meter box model called by the plan is specifically:

[0024] Each element in the summary set of each defect detection area in the called meter box model corresponds to a defect detection area. The abnormal defect detection results of the called meter box model in the historical data include the ratio of the abnormal frequency corresponding to each defect detection area in the historical detection data to the total defect detection frequency of the corresponding model meter box; when planning the grid interval density within each defect detection area in the called meter box model, the calculation formula involved is as follows:

[0025] M i =(r1·B i +r2·N i +r3·P i )·L

[0026] Among them, M i represents the grid spacing density in the i-th defect detection area in the metering box model called by the plan; L represents the preset grid line spacing of the grid planning; B i N represents the ratio of the abnormal frequency corresponding to the i-th defect detection area in the abnormal defect detection results of the called meter box model in the historical data to the total defect detection frequency of the corresponding model meter box; i represents the number of defect detection areas adjacent to and continuous with the i-th defect detection area in the called meter box model; r1, r2 and r3 represent the preset first weight coefficient, second weight coefficient and third weight coefficient respectively; P i It represents the regional flatness within the i-th defect detection area in the called metering box model. The calculation formula is as follows:

[0027] P i =min{H1 / H2, H3 / H1}

[0028] Wherein, H1 represents the average length of each position point in the i-th defect detection area of ​​the called meter box model from the corresponding reference plane, and each defect detection area in the called meter box model corresponds to a preset reference plane; H2 represents the average length of each position point in the i-th defect detection area of ​​the called meter box model whose distance to the corresponding reference plane is greater than H1 from the corresponding reference plane; H3 represents the average length of each position point in the i-th defect detection area of ​​the called meter box model whose distance to the corresponding reference plane is less than H1 from the corresponding reference plane; min{} represents the operation of finding the minimum value; when H1=H2 and H1=H3, it is determined that P i =0;

[0029] The grid line positions in the defect detection area of ​​the meter box model determined based on the grid spacing density are specifically as follows: from left to right, from front to back, and from top to bottom, based on the grid spacing density M i Determine the positions of each grid line within the i-th defect detection area in the called meter box model.

[0030] Furthermore, in S4, the viewing angle feature includes the viewing angle vector corresponding to each camera, the light intensity monitored at each camera position, and the color number corresponding to each defect detection area in the best-captured image corresponding to each camera; the lighting feature includes the lighting direction relative to the center point of the meter box model and the maximum light intensity at the meter box model; the color number corresponding to the defect detection area is the average value of the color numbers corresponding to each pixel point in the corresponding defect detection area; the first parameter in the relationship mapping data pair represents the viewing angle feature, and the second parameter represents the lighting feature;

[0031] The matching of the illumination characteristics of the meter box to be tested is specifically as follows: obtaining the viewing angle characteristics of the camera corresponding to each element in the multi-view optimal acquisition image set of the meter box to be tested, and mapping the first parameter to the viewing angle characteristics of the camera corresponding to each element in the multi-view optimal acquisition image set of the meter box to be tested as the second parameter in the relationship mapping data pair having the same relationship as the first parameter as the illumination characteristics of the meter box to be tested;

[0032] The combined pose mapping information of the called meter box model is used to generate the projection area of ​​each defect detection area in the called meter box model based on the viewing angle of each camera, specifically:

[0033] Combined with the pose mapping information of the called meter box model, and adjusting the lighting characteristics of the scene in which the called meter box model is located to the lighting characteristics of the meter box to be tested, simulate the projection area of ​​each defect detection area in the called meter box model based on the viewing angle of each camera to obtain a simulated image, and distinguish the projection area of ​​each grid area in each defect detection area in the called meter box model based on the viewing angle of each camera;

[0034] The mapping area of ​​the viewing angle projection area in the image captured by the corresponding camera in the multi-view optimal captured image set of the measuring box to be measured is the same as the position area of ​​the corresponding viewing angle projection area in the simulation image.

[0035] Furthermore, in S5, the formula for calculating the color number deviation coefficient between each viewing angle projection area and the corresponding mapping area is specifically:

[0036]

[0037] Among them, g i K represents the color deviation coefficient between the viewing angle projection area of ​​the i-th defect detection area and the corresponding mapping area; i Represents the number of unobstructed grid areas within the viewing angle projection area of ​​the i-th defect detection area;

[0038] The set consisting of the kth unmasked grid area in the viewing angle projection area of ​​the i-th defect detection area and the remaining unmasked grid areas where the corresponding grid boundary line overlaps with the grid boundary line of the kth unmasked grid area is recorded as W (i，k) ; The grid boundary lines represent the grid line segments corresponding to each edge of the corresponding grid; ET (i，k) W (i，k) Each pixel in the image corresponds to the average value of the color number, EY (i，k) W (i，k) The average value of the color number corresponding to each pixel in the mapping area corresponding to the inner element; (i，k) , EY (i，k)} indicates ET (i，k) with EY (i，k) The corresponding deviation factor in the database preset table.

[0039] Furthermore, in S5, the defect detection feedback information of the meter box to be tested is generated based on the color number deviation coefficient as follows:

[0040] When the maximum value of the color number deviation coefficient between the corresponding perspective projection area and the corresponding mapping area of ​​the same defect detection area in the metering box to be tested based on each camera perspective is greater than a preset value, the detection result of the corresponding defect detection area in the metering box to be tested is judged to be abnormal; otherwise, the detection result of the corresponding defect detection area in the metering box to be tested is judged to be normal.

[0041] The present invention also proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-described adaptive detection method for defects in a metering box based on multi-vision is implemented.

[0042] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the above-mentioned multi-vision-based adaptive detection method for measuring box defects.

[0043] The beneficial effects of the present invention are:

[0044] (1) The present invention uses multi-view vision to obtain images of the meter box at various viewing angles. In this process, the deviation of the camera position and shooting angle due to camera vibration may cause the meter box position in the captured image to be abnormal. Therefore, the accuracy of the subsequent defect detection results is ensured by screening the best captured images of each camera, and the feature identification of the meter box is extracted and identified by OCR technology, which provides data support for the subsequent construction of the posture mapping information of the call model;

[0045] (2) The present invention combines the abnormal detection results of the meter box model in the historical data to realize the grid spacing density in each defect detection area in the meter box model, providing a basis for the subsequent analysis of the color number deviation coefficient of each defect detection area, and realizes the adaptive adjustment of the analysis accuracy of the color number deviation coefficient of each defect detection area by dynamically adjusting the grid spacing density in the defect detection area (the higher the grid spacing density in the defect detection area, the higher the analysis accuracy of the color number deviation coefficient of the corresponding defect detection area). BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of the adaptive detection method for measuring box defects based on multi-vision proposed by the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] Example 1

[0049] The present invention proposes a method for self-adaptive detection of defects in meter boxes based on multi-vision. The overall process of the method is as follows: Figure 1 As shown, the following steps are included:

[0050] S1. Synchronously capture multi-view images of the meter box to be tested using multiple cameras, and construct a multi-view visual data analysis set for the meter box to be tested based on the preset positions of each camera; and select the optimal multi-view image set for the meter box to be tested from the multi-view visual data analysis set;

[0051] In S1, the multi-view visual data analysis set of the meter box to be tested includes multi-view images of the meter box to be tested that are synchronously acquired by multiple cameras at different time points, and the image of the meter box to be tested acquired by each camera is regarded as a view image of the meter box to be tested;

[0052] The specific process of selecting the optimal multi-view image set of the meter box to be tested from the multi-view visual data analysis set is as follows:

[0053] The images of the meter box to be measured collected by the same camera at different time points are obtained respectively, and the pixel point of the center point of the meter box area to be measured in the image of the meter box to be measured is obtained, and recorded as the perspective reference point of the image of the meter box to be measured; the summary set of the perspective reference points of the images of the meter box to be measured collected by the same camera at different time points is recorded as the perspective reference point set of the corresponding camera; the element with the smallest sum of pixel distances from the remaining elements in the perspective reference point set of the corresponding camera is recorded as the perspective calibration point; any image of the meter box to be measured whose corresponding perspective reference point coincides with the perspective calibration point in the images of the meter box to be measured collected by the corresponding camera at different time points is used as the best perspective collected image of the meter box to be measured based on the corresponding camera, and the best perspective collected images of all cameras are combined into a multi-perspective best collection image set.

[0054] The present invention takes into account that the fluctuation deviation of the camera's viewing angle and fixed position due to vibration during the shooting process often affects the collected meter box image, and thus causes large errors in the color number deviation analysis results of each defect detection area in the subsequent process; therefore, it is necessary to screen the images collected by the viewing angle of each camera to provide data support for the subsequent judgment of each defect detection area and analysis of the color number deviation of the corresponding defect detection area.

[0055] S2. Extract feature identifiers of each element in the multi-view optimal acquisition image set of the meter box to be tested by using OCR technology, and compare the feature identifier extraction results with the preset table in the database to obtain the model of the meter box to be tested in the preset table in the database, and call the meter box model with this model in the database; based on the preset position of the camera to which each element in the multi-view optimal acquisition image set of the meter box to be tested belongs and the corresponding image acquisition view angle vector, construct the posture mapping information of the called meter box model.

[0056] The specific process of extracting feature identification of each element in the multi-view optimal acquisition image set of the meter box to be measured using OCR technology is as follows:

[0057] The image in the sliding window area is compared with the preset identifier respectively by using a sliding window method, and the feature identifier of the sliding window area whose similarity with the preset identifier is greater than a preset value is used as the corresponding feature identifier extraction result; the size of the sliding window is preset, and the similarity between the feature identifier of the sliding window area and the preset identifier is calculated by: proportionally enlarging the feature identifier extraction result of the sliding window area and making it equal to the length of the corresponding preset identifier, and the quotient of the number of overlapping pixels between the feature identifier extraction result of the sliding window area after the proportional enlargement and the preset identifier is divided by the number of pixels of the preset identifier as the similarity.

[0058] In the present invention, the feature identification extraction results of the sliding window area are proportionally enlarged and made the same length as the corresponding preset identification. This is because the distance between the camera and the meter box to be measured will affect the size of the pixel area corresponding to the meter box to be measured in the captured image, which will lead to the area size of the feature identification extraction results of the sliding window area being non-uniform. The proportional scaling is to facilitate its comparison with the preset identification.

[0059] Compare the feature identification extraction results with the database preset table to obtain the model of the meter box to be tested in the database preset table:

[0060] When comparing the feature identification extraction results with the database preset table, the meter box model with the most overlapping feature identification extraction results is used as the model corresponding to the meter box to be tested;

[0061] The vector direction of the image acquisition perspective vector is the direction from the corresponding camera position to the center point of the corresponding perspective acquisition image, and the vector modulus is 1; the posture mapping information of the called meter box model includes the preset position of the camera belonging to each element in the multi-perspective optimal acquisition image set of the meter box to be measured and the corresponding image acquisition perspective vector.

[0062] S3. By querying the summary set of each defect detection area in the called meter box model, combined with the abnormal defect detection results of the called meter box model in the historical data, plan the grid interval density in each defect detection area in the called meter box model, and determine the position of each grid line in the defect detection area in the called meter box model based on the grid interval density.

[0063] The grid spacing density within each defect detection area in the metering box model called by the plan is specifically:

[0064] Each element in the summary set of each defect detection area in the called meter box model corresponds to a defect detection area. The abnormal defect detection results of the called meter box model in the historical data include the ratio of the abnormal frequency corresponding to each defect detection area in the historical detection data to the total defect detection frequency of the corresponding model meter box; when planning the grid interval density within each defect detection area in the called meter box model, the calculation formula involved is as follows:

[0065] M i =(r1·B i +r2·N i +r3·P i )·L

[0066] Among them, M i represents the grid spacing density in the i-th defect detection area in the metering box model called by the plan; L represents the preset grid line spacing of the grid planning; B i N represents the ratio of the abnormal frequency corresponding to the i-th defect detection area in the abnormal defect detection results of the called meter box model in the historical data to the total defect detection frequency of the corresponding model meter box; i represents the number of defect detection areas adjacent to and continuous with the i-th defect detection area in the called meter box model; r1, r2 and r3 represent the preset first weight coefficient, second weight coefficient and third weight coefficient respectively; P i It represents the regional flatness within the i-th defect detection area in the called metering box model. The calculation formula is as follows:

[0067] P i =min{H1 / H2, H3 / H1}

[0068] Wherein, H1 represents the average length of each position point in the i-th defect detection area of ​​the called meter box model from the corresponding reference plane, and each defect detection area in the called meter box model corresponds to a preset reference plane; H2 represents the average length of each position point in the i-th defect detection area of ​​the called meter box model whose distance to the corresponding reference plane is greater than H1 from the corresponding reference plane; H3 represents the average length of each position point in the i-th defect detection area of ​​the called meter box model whose distance to the corresponding reference plane is less than H1 from the corresponding reference plane; min{} represents the operation of finding the minimum value; when H1=H2 and H1=H3, it is determined that P i =0;

[0069] The grid line positions in the defect detection area of ​​the meter box model determined based on the grid spacing density are specifically as follows: from left to right, from front to back, and from top to bottom, based on the grid spacing density M iDetermine the positions of each grid line within the i-th defect detection area in the called meter box model.

[0070] S4. Obtain the viewing angle characteristics and illumination characteristics of the meter box model with normal detection results corresponding to the identified model, and the viewing angle characteristics and illumination characteristics constitute a relationship mapping data pair; use the viewing angle characteristics of the camera corresponding to each element in the multi-view optimal acquisition image set of the meter box to be tested to match the illumination characteristics of the meter box to be tested; and combine the posture mapping information of the called meter box model to generate the viewing angle projection area of ​​each defect detection area in the called meter box model based on each camera, as well as the mapping area of ​​the viewing angle projection area in the image acquired by the corresponding camera in the multi-view optimal acquisition image set of the meter box to be tested.

[0071] The viewing angle features include the viewing angle vector corresponding to each camera, the light intensity monitored at each camera position, and the color number corresponding to each defect detection area in the optimal image captured by each camera. The lighting features include the light direction relative to the center point of the meter box model and the maximum light intensity at the meter box model. The color number corresponding to the defect detection area is the average value of the color numbers corresponding to each pixel in the corresponding defect detection area. The first parameter in the relationship mapping data pair represents the viewing angle feature, and the second parameter represents the lighting feature.

[0072] Matching the illumination characteristics of the meter box to be tested specifically comprises: obtaining the viewing angle characteristics of the camera corresponding to each element in the multi-view optimal acquisition image set of the meter box to be tested, and mapping the first parameter to the second parameter in the data pair having the same relationship as the viewing angle characteristics of the camera corresponding to each element in the multi-view optimal acquisition image set of the meter box to be tested as the illumination characteristics of the meter box to be tested;

[0073] Combined with the pose mapping information of the called meter box model, the projection area of ​​each defect detection area in the called meter box model based on the viewing angle of each camera is generated as follows:

[0074] Combined with the pose mapping information of the called meter box model, and adjusting the lighting characteristics of the scene in which the called meter box model is located to the lighting characteristics of the meter box to be tested, simulate the projection area of ​​each defect detection area in the called meter box model based on the viewing angle of each camera to obtain a simulated image, and distinguish the projection area of ​​each grid area in each defect detection area in the called meter box model based on the viewing angle of each camera;

[0075] The mapping area of ​​the viewing angle projection area in the image captured by the corresponding camera in the multi-view optimal captured image set of the metering box to be measured is the same as the position area of ​​the corresponding viewing angle projection area in the simulation image.

[0076] S5. Each defect detection area in the called meter box model is composed of the unmasked grid area and the masked grid area in the corresponding viewing angle projection area; the unmasked grid area and the positional relationship between each unmasked grid area in the viewing angle projection area are obtained, and the color number deviation coefficient between each viewing angle projection area and the corresponding mapping area is calculated, and defect detection feedback information of the meter box to be tested is generated based on the color number deviation coefficient.

[0077] In this embodiment, the relationship between the adjacent unobstructed grid areas in the viewing angle projection area and the corresponding grid areas in the called meter box model may be adjacent, or they may be separated by the remaining grid areas (non-adjacent positions); for example, different positions in a defect detection area are uneven, and due to the camera shooting angle, the high-position area in the captured image may cover the low-position area, and then the adjacent grid areas that are not obscured in the acquired image may be non-adjacent in the meter box model, and the corresponding low-position areas between the two are obscured and not presented in the acquired image.

[0078] The formula for calculating the color deviation coefficient between each viewing angle projection area and the corresponding mapping area is as follows:

[0079]

[0080] Among them, g i K represents the color deviation coefficient between the viewing angle projection area of ​​the i-th defect detection area and the corresponding mapping area; i Represents the number of unobstructed grid areas within the viewing angle projection area of ​​the i-th defect detection area;

[0081] The set consisting of the kth unmasked grid area in the viewing angle projection area of ​​the i-th defect detection area and the remaining unmasked grid areas where the corresponding grid boundary line overlaps with the grid boundary line of the kth unmasked grid area is recorded as W (i，k) ; In this embodiment, the remaining unmasked grid areas where the corresponding grid boundary lines overlap with the grid boundary lines of the kth unmasked grid area may belong to the i-th defect detection area, or may belong to a defect detection area adjacent to the i-th defect detection area, or may even belong to a defect detection area that is not adjacent to the i-th defect detection area.

[0082] The grid boundary lines represent the grid line segments corresponding to each edge of the corresponding grid; ET (i，k) W (i，k) Each pixel in the image corresponds to the average value of the color number, EY (i，k) W (i，k) The average value of the color number corresponding to each pixel in the mapping area corresponding to the inner element;(i，k) , EY (i，k)} indicates ET (i，k) with EY (i，k) The corresponding deviation factor in the database preset table.

[0083] The defect detection feedback information of the meter box to be tested is generated based on the color number deviation coefficient as follows:

[0084] When the maximum value of the color number deviation coefficient between the corresponding perspective projection area and the corresponding mapping area of ​​the same defect detection area in the metering box to be tested based on each camera perspective is greater than a preset value, the detection result of the corresponding defect detection area in the metering box to be tested is judged to be abnormal; otherwise, the detection result of the corresponding defect detection area in the metering box to be tested is judged to be normal.

[0085] Example 2

[0086] The present invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for adaptively detecting defects in a metering box based on multi-viewing as described in Example 1 is implemented.

[0087] Example 3

[0088] The present invention provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the multi-vision-based adaptive detection method for measuring box defects as described in the first embodiment.

[0089] In the embodiments disclosed herein, computer storage media can be tangible media that can contain or store programs for use by or in conjunction with an instruction execution system, device, or apparatus. Computer storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of computer storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0090] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0091] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for adaptively detecting defects in a metering box based on multi-viewing, characterized in that: The following steps are involved: S1. Synchronously capture multi-view images of the meter box to be tested using multiple cameras, and construct a multi-view visual data analysis set for the meter box to be tested based on the preset positions of each camera; and select the optimal multi-view image set for the meter box to be tested from the multi-view visual data analysis set; S2. Extract feature identifiers from each element in the multi-view optimally captured image set of the meter box to be tested using optical character recognition (OCR) technology, compare the feature identifier extraction results with a preset table in a database, obtain the model of the meter box to be tested in the preset table in the database, and call a meter box model with that model in the database; construct pose mapping information for the called meter box model based on the preset position of the camera to which each element in the multi-view optimally captured image set belongs and the corresponding image acquisition view angle vector; S3. Querying the summary set of each defect detection area in the called meter box model and combining the abnormal defect detection results of the called meter box model in historical data, plan the grid spacing density within each defect detection area in the called meter box model, and determine the position of each grid line within the defect detection area in the called meter box model based on the grid spacing density; S4. Obtain the viewing angle characteristics and illumination characteristics of the meter box model with normal detection results corresponding to the identified model, wherein the viewing angle characteristics and illumination characteristics constitute a relationship mapping data pair; use the viewing angle characteristics of the camera corresponding to each element in the multi-view optimal acquisition image set of the meter box to be tested to match the illumination characteristics of the meter box to be tested; Combined with the pose mapping information of the called meter box model, the view projection area of ​​each defect detection area in the called meter box model based on each camera is generated, as well as the mapping area of ​​the view projection area in the image captured by the corresponding camera in the multi-view optimal capture image set of the meter box to be tested; S5. Obtain the unobstructed grid areas within the viewing angle projection area, as well as the positional relationship between each unobstructed grid area within the viewing angle projection area, and calculate the color number deviation coefficient between each viewing angle projection area and the corresponding mapping area, and generate defect detection feedback information of the meter box to be tested based on the color number deviation coefficient.

2. The method for adaptively detecting defects in a measuring box based on multi-viewing according to claim 1, characterized in that: In S1, the multi-view visual data analysis set of the meter box to be tested includes multi-view images of the meter box to be tested that are synchronously acquired by multiple cameras at different time points, and the image of the meter box to be tested acquired by each camera is regarded as a view image of the meter box to be tested; The specific process of selecting the optimal multi-view image set of the meter box to be tested from the multi-view visual data analysis set of the meter box to be tested is as follows: The images of the meter box to be measured collected by the same camera at different time points are obtained respectively, and the pixel point of the center point of the meter box area to be measured in the image of the meter box to be measured is obtained, and recorded as the perspective reference point of the image of the meter box to be measured; the summary set of the perspective reference points of the images of the meter box to be measured collected by the same camera at different time points is recorded as the perspective reference point set of the corresponding camera; the element with the smallest sum of pixel distances from the remaining elements in the perspective reference point set of the corresponding camera is recorded as the perspective calibration point; any image of the meter box to be measured whose corresponding perspective reference point coincides with the perspective calibration point in the images of the meter box to be measured collected by the corresponding camera at different time points is used as the best perspective collected image of the meter box to be measured based on the corresponding camera, and the best perspective collected images of all cameras are combined into a multi-perspective best collection image set.

3. The method for adaptively detecting defects in a measuring box based on multi-viewing according to claim 1, characterized in that: In S2, the specific process of extracting feature identification of each element in the multi-view optimally collected image set of the measuring box to be measured by using the OCR technology is as follows: The image in the sliding window area is compared with the preset identifier respectively by using a sliding window method, and the feature identifier of the sliding window area whose similarity with the preset identifier is greater than a preset value is used as the corresponding feature identifier extraction result; the size of the sliding window is preset, and the similarity between the feature identifier of the sliding window area and the preset identifier is calculated by: proportionally enlarging the feature identifier extraction result of the sliding window area and making it equal to the length of the corresponding preset identifier, and the quotient of the number of overlapping pixels between the feature identifier extraction result of the sliding window area after the proportional enlargement and the preset identifier is divided by the number of pixels of the preset identifier as the similarity.

4. The method for adaptively detecting defects in a measuring box based on multi-viewing according to claim 1, characterized in that: In S2, the feature identification extraction result is compared with the database preset table to obtain the model of the meter box to be tested in the database preset table: When comparing the feature identification extraction results with the database preset table, the meter box model with the most overlapping feature identification extraction results is used as the model corresponding to the meter box to be tested; The vector direction of the image acquisition perspective vector is the direction from the corresponding camera position to the center point of the corresponding perspective acquisition image, and the vector modulus is 1; the posture mapping information of the called meter box model includes the preset position of the camera belonging to each element in the multi-perspective optimal acquisition image set of the meter box to be measured and the corresponding image acquisition perspective vector.

5. The method for adaptively detecting defects in a measuring box based on multi-viewing according to claim 1, characterized in that: In S3, the grid spacing density in each defect detection area in the meter box model called by the plan is specifically: Each element in the summary set of each defect detection area in the called meter box model corresponds to a defect detection area. The abnormal defect detection results of the called meter box model in the historical data include the ratio of the abnormal frequency corresponding to each defect detection area in the historical detection data to the total defect detection frequency of the corresponding model meter box; when planning the grid interval density within each defect detection area in the called meter box model, the calculation formula involved is as follows: M i =(r1·B i +r2·N i +r3·P i )·L Among them, M i represents the grid spacing density in the i-th defect detection area in the metering box model called by the plan; L represents the preset grid line spacing of the grid planning; B i N represents the ratio of the abnormal frequency corresponding to the i-th defect detection area in the abnormal defect detection results of the called meter box model in the historical data to the total defect detection frequency of the corresponding model meter box; i represents the number of defect detection areas adjacent to and continuous with the i-th defect detection area in the called meter box model; r1, r2 and r3 represent the preset first weight coefficient, second weight coefficient and third weight coefficient respectively; P i It represents the regional flatness within the i-th defect detection area in the called metering box model. The calculation formula is as follows: P i =min{H1 / H2,H3 / H1} Wherein, H1 represents the average length of each position point in the i-th defect detection area of ​​the called meter box model from the corresponding reference plane, and each defect detection area in the called meter box model corresponds to a preset reference plane; H2 represents the average length of each position point in the i-th defect detection area of ​​the called meter box model whose distance to the corresponding reference plane is greater than H1 from the corresponding reference plane; H3 represents the average length of each position point in the i-th defect detection area of ​​the called meter box model whose distance to the corresponding reference plane is less than H1 from the corresponding reference plane; min{} represents the operation of finding the minimum value; when H1=H2 and H1=H3, it is determined that P i =0; The grid line positions in the defect detection area of ​​the meter box model determined based on the grid spacing density are specifically as follows: from left to right, from front to back, and from top to bottom, based on the grid spacing density M i Determine the positions of each grid line within the i-th defect detection area in the called meter box model.

6. The method for adaptively detecting defects in a measuring box based on multi-viewing according to claim 1, characterized in that: In S4, the viewing angle features include the viewing angle vector corresponding to each camera, the light intensity monitored at each camera position, and the color numbers corresponding to each defect detection area in the image best captured by the viewing angle corresponding to each camera; The illumination characteristics include the illumination direction relative to the center point of the meter box model and the maximum illumination intensity at the meter box model; the color number corresponding to the defect detection area is the average value of the color numbers corresponding to each pixel in the corresponding defect detection area; the first parameter in the relationship mapping data pair represents the viewing angle characteristic, and the second parameter represents the illumination characteristic; The matching of the illumination characteristics of the meter box to be tested is specifically as follows: obtaining the viewing angle characteristics of the camera corresponding to each element in the multi-view optimal acquisition image set of the meter box to be tested, and mapping the first parameter to the viewing angle characteristics of the camera corresponding to each element in the multi-view optimal acquisition image set of the meter box to be tested as the second parameter in the relationship mapping data pair having the same relationship as the first parameter as the illumination characteristics of the meter box to be tested; The combined pose mapping information of the called meter box model is used to generate the projection area of ​​each defect detection area in the called meter box model based on the viewing angle of each camera, specifically: Combined with the pose mapping information of the called meter box model, and adjusting the lighting characteristics of the scene in which the called meter box model is located to the lighting characteristics of the meter box to be tested, simulate the projection area of ​​each defect detection area in the called meter box model based on the viewing angle of each camera to obtain a simulated image, and distinguish the projection area of ​​each grid area in each defect detection area in the called meter box model based on the viewing angle of each camera; The mapping area of ​​the viewing angle projection area in the image captured by the corresponding camera in the multi-view optimal captured image set of the measuring box to be measured is the same as the position area of ​​the corresponding viewing angle projection area in the simulation image.

7. The method for adaptively detecting defects in a measuring box based on multi-viewing according to claim 1, characterized in that: In S5, the formula for calculating the color number deviation coefficient between each viewing angle projection area and the corresponding mapping area is specifically: Among them, g i K represents the color deviation coefficient between the viewing angle projection area of ​​the i-th defect detection area and the corresponding mapping area; i Represents the number of unobstructed grid areas within the viewing angle projection area of ​​the i-th defect detection area; The set consisting of the kth unmasked grid area in the viewing angle projection area of ​​the i-th defect detection area and the remaining unmasked grid areas where the corresponding grid boundary line overlaps with the grid boundary line of the kth unmasked grid area is recorded as W (i，k) ; The grid boundary lines represent the grid line segments corresponding to each edge of the corresponding grid; ET (i，k) W (i，k) Each pixel in the image corresponds to the average value of the color number, EY (i，k) W (i，k) The average value of the color number corresponding to each pixel in the mapping area corresponding to the inner element; (i，k) , EY (i，k) } indicates ET (i，k) with EY (i，k) The corresponding deviation factor in the database preset table.

8. The method for adaptively detecting defects in a measuring box based on multi-viewing according to claim 1, characterized in that: In S5, the defect detection feedback information of the measuring box to be tested is generated based on the color number deviation coefficient as follows: When the maximum value of the color number deviation coefficient between the corresponding perspective projection area and the corresponding mapping area of ​​the same defect detection area in the measured meter box is greater than the preset value based on the perspective of each camera, the detection result of the corresponding defect detection area in the measured meter box is determined to be abnormal; Otherwise, the detection result of the corresponding defect detection area in the meter box to be tested is determined to be normal.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for adaptively detecting defects in a metering box based on multi-vision according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: The computer program enables the computer to execute the multi-vision-based adaptive detection method for measuring box defects as described in any one of claims 1 to 8.

Images