Intelligent detection methods, devices, equipment, and media based on image recognition

Abstract

This invention discloses an intelligent detection method, apparatus, device, and medium based on image recognition. The method includes: segmenting an optical image acquired by a camera device to obtain a chip image for each chip; extracting image feature information from each chip image and performing anomaly identification, integrity identification, and wiring identification; determining the anomaly type of the abnormal chip image based on the anomaly identification results, integrity identification results, and wiring identification results; and combining the anomaly coordinate position and anomaly type of the abnormal chip image to obtain anomaly detection information. This method can segment an optical image to obtain a chip image for each chip, identify each chip image individually to determine whether it is an abnormal chip image, and combine the anomaly coordinate position and anomaly type of the abnormal chip image to obtain anomaly detection information, significantly improving the efficiency and reliability of detecting the assembly quality of sensor chips.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection technology, and in particular to an intelligent detection method, apparatus, device, and medium based on image recognition. Background Technology

[0002] In sensor manufacturing, chips need to be assembled in predetermined positions on large-size circuit boards and electrically connected to other components. To inspect the assembly quality, manual checks are required to identify any abnormalities in the sensor chips assembled on the entire circuit board. Sensors in abnormal positions are then cut off and discarded. However, current techniques, which involve illuminating the circuit board with a magnifying glass and manually inspecting it, cannot quickly assess the assembly quality of the sensor chips according to a unified standard, thus affecting the efficiency and accuracy of sensor chip inspection. Therefore, existing methods suffer from the inability to quickly inspect the assembly quality of sensor chips on circuit boards. Summary of the Invention

[0003] This invention provides an intelligent detection method, apparatus, device, and medium based on image recognition, aiming to solve the problem in existing technologies that cannot quickly detect the assembly quality of sensor chips on circuit boards.

[0004] In a first aspect, embodiments of the present invention provide an intelligent detection method based on image recognition. The method is applied in a detection device, which is connected to a camera via a network to transmit data information. The method includes:

[0005] If an optical image acquired by the camera device is received, the optical image is segmented according to a preset image segmentation rule to obtain a chip image corresponding to each chip in the optical image;

[0006] The corresponding image feature information is extracted from each chip image according to the preset image feature extraction model;

[0007] Based on the preset component anomaly identification rules, anomaly identification is performed on each of the image feature information to obtain the component anomaly identification result.

[0008] If the component anomaly identification result of the image feature information is not abnormal, the image feature information is subjected to integrity identification according to the preset integrity identification rules to obtain the integrity identification result of whether it is complete;

[0009] If the integrity recognition result of the image feature information is complete, the image feature information is subjected to wiring recognition according to the preset wiring recognition rules to obtain the wiring recognition result of whether the wiring is abnormal;

[0010] The abnormal chip image and the corresponding abnormal type are determined based on the component anomaly identification result, the integrity identification result, and the wiring identification result.

[0011] The abnormal chip images are combined with the abnormal coordinates of the optical image and the corresponding abnormal type to obtain the abnormal detection information corresponding to the optical image.

[0012] Secondly, embodiments of the present invention also provide an intelligent detection device based on image recognition, wherein the device is used to execute the intelligent detection method based on image recognition as described in the first aspect above, the device is configured in a detection device, the detection device is connected to a camera device via a network to realize the transmission of data information, and the device includes:

[0013] An image segmentation unit is used to segment an optical image acquired by the camera device according to a preset image segmentation rule to obtain a chip image corresponding to each chip in the optical image if the optical image is received.

[0014] The image feature information acquisition unit is used to extract the corresponding image feature information from each chip image according to the preset image feature extraction model;

[0015] The first identification unit is used to perform anomaly identification on each of the image feature information according to the preset component anomaly identification rules, and obtain the component anomaly identification result of whether it is abnormal.

[0016] The second identification unit is used to perform integrity identification on the image feature information according to a preset integrity identification rule if the component anomaly identification result of the image feature information is not abnormal, and obtain an integrity identification result of whether it is complete.

[0017] The third identification unit is used to identify the wiring of the image feature information according to the preset wiring identification rules if the integrity identification result of the image feature information is complete, and to obtain the wiring identification result of whether the wiring is abnormal.

[0018] The determining unit is used to determine the abnormal chip image and the corresponding abnormal type based on the component abnormality identification result, the integrity identification result and the wiring identification result;

[0019] The combination unit is used to combine the abnormal chip images of each abnormal chip with the abnormal coordinate position of the optical image and the corresponding abnormal type to obtain the abnormal detection information corresponding to the optical image.

[0020] Thirdly, embodiments of the present invention also provide a computer device, wherein the device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0021] Memory, used to store computer programs;

[0022] When a processor executes a program stored in memory, it implements the steps of the image recognition-based intelligent detection method described in the first aspect above.

[0023] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the image recognition-based intelligent detection method described in the first aspect above.

[0024] This invention provides an intelligent detection method, apparatus, device, and medium based on image recognition. The method includes: segmenting an optical image acquired by a camera device to obtain a chip image for each chip; extracting image feature information from each chip image and performing anomaly identification, integrity identification, and wiring identification; determining the anomaly type of the abnormal chip image based on the anomaly identification results, integrity identification results, and wiring identification results; and combining the anomaly coordinate position and anomaly type of the abnormal chip image to obtain anomaly detection information. This method can segment an optical image to obtain a chip image for each chip, identify each chip image individually to determine whether it is an abnormal chip image, and obtain anomaly detection information by combining the anomaly coordinate position and anomaly type of the abnormal chip image, significantly improving the efficiency and reliability of detecting the assembly quality of sensor chips. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A flowchart illustrating the intelligent detection method based on image recognition provided in an embodiment of the present invention;

[0027] Figure 2 This is a schematic diagram illustrating an application scenario of the image recognition-based intelligent detection method provided in an embodiment of the present invention.

[0028] Figure 3 A schematic block diagram of an image recognition-based intelligent detection device provided in an embodiment of the present invention;

[0029] Figure 4 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0032] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0033] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0034] Please see Figure 1 As shown in the figure, an embodiment of this invention provides an intelligent detection method based on image recognition. This method is applied to a detection device 10 and is executed by application software installed in the detection device 10. The detection device 10 is network-connected to a camera device 20 to transmit data. The camera device 20 is used to magnify and acquire images of the circuit board; it can acquire optical images of the circuit board. The detection device 10 is a terminal device, such as a desktop computer, laptop computer, tablet computer, or mobile phone, used to detect and recognize the received optical images. Figure 1 As shown, the method includes steps S110 to S170.

[0035] S110. If an optical image acquired by the camera device is received, the optical image is segmented according to a preset image segmentation rule to obtain a chip image corresponding to each chip in the optical image.

[0036] If an optical image acquired by the camera device is received, the optical image is segmented according to a preset image segmentation rule to obtain a chip image corresponding to each chip in the optical image. If the camera device acquires an optical image of the circuit board, it can send the optical image to the detection device. Since the circuit board contains multiple sensor chips, the optical image needs to be segmented using image segmentation rules to obtain a chip image corresponding to each chip, and each chip image is then individually identified and detected.

[0037] In a specific embodiment, step S110 includes the following sub-steps: determining the corresponding template boundary position and template boundary size based on the positioning mark on the optical image; adjusting the default cutting template in the image segmentation rules according to the template boundary size to obtain a target cutting template that matches the template boundary size; covering the upper layer of the optical image with the target cutting template according to the template boundary position; and cutting the optical image according to the target cutting template to obtain a chip image.

[0038] Specifically, if positioning marks are affixed / etched on the circuit board, then the optical image will also contain these positioning marks. The optical image can contain multiple positioning marks, such as positioning marks at the four top corners of the circuit board, or at two diagonally opposite corners. Since the positioning marks are located at fixed positions on the circuit board, the location of the outer frame enclosed by the positioning marks can be determined as the template boundary position by the distance between the positioning marks and their positions. The boundary dimensions of the outer frame are then calculated based on the distance between the positioning marks and used as the template boundary dimensions.

[0039] Furthermore, the default cutting template is scaled and adjusted according to the template boundary dimensions to match the adjusted target cutting template with the template boundary dimensions. Based on the template boundary position, the target cutting template is placed over the optical image and cuts the optical image. The target cutting template contains multiple meridians and parallels, which are used to cut the optical image, thus obtaining the chip images corresponding to each chip.

[0040] In a specific embodiment, after cutting the optical image according to the target cutting template to obtain the chip image, the method further includes: obtaining a corresponding QR code image from the QR code image cutting area set in the target cutting template; and recognizing the QR code image according to a preset QR code recognition model to obtain QR code information corresponding to the optical image.

[0041] Specifically, the target cutting template also includes a QR code image cutting area. The QR code image corresponding to this cutting area can be obtained and recognized using a QR code recognition model. The QR code image can be a barcode or a scattered QR code. The corresponding recognition template in the QR code recognition model can then be used to recognize different types of QR code images, thereby obtaining QR code information. This QR code information is used to uniquely identify the circuit board currently being inspected.

[0042] S120. Extract the corresponding image feature information from each chip image according to the preset image feature extraction model.

[0043] Image feature information is extracted from each chip image based on a pre-defined image feature extraction model. The image feature extraction model is the specific model used to extract image features from the chip image.

[0044] In a specific embodiment, step S120 includes the following sub-steps: extracting region pixel features from each of the chip images according to the feature extraction region in the image feature extraction model to obtain region pixel feature information corresponding to each of the chip images; extracting feature pixel contours from each of the chip images according to the feature pixel interval in the image feature extraction model to obtain feature pixel contour information corresponding to each of the chip images; and combining the region pixel feature information and the feature pixel contour information to form corresponding image feature information.

[0045] The image feature extraction model contains multiple feature extraction regions, each corresponding to a specific region where a component is located. Based on the extraction location of each feature extraction region, the pixel values ​​of the pixels within that region are extracted from each chip image, serving as the regional pixel features of that region. The regional pixel features of each feature extraction region in the chip image are then combined to form the regional pixel feature information of the chip image. Furthermore, the image feature extraction model also includes multiple feature pixel intervals, each used to define a range of pixel values. It can then determine whether the pixel values ​​of each pixel in the chip image are within a feature pixel interval, thereby obtaining all pixels within a feature pixel interval and extracting the outer contour of the region formed by all pixels, thus obtaining the feature pixel contour of that feature pixel interval. The feature pixel contours of each feature pixel interval in the chip image are then combined to form the feature pixel contour information of the chip image.

[0046] By combining the regional pixel feature information and the feature pixel contour information of the chip image, the image feature information of the chip image can be obtained.

[0047] S130. Based on the preset component anomaly identification rules, perform anomaly identification on each of the image feature information to obtain the component anomaly identification result.

[0048] According to the preset component anomaly identification rules, anomaly identification is performed on each of the image feature information to obtain a component anomaly identification result. Further, the component anomaly identification rules can be used to identify whether there are anomalies in the components in the chip image. Specifically, the anomaly identification rules can be used to identify anomalies in the image feature information of each chip image. Anomaly identification is to identify whether there are missing MEMS components, missing ASIC components, locally damaged films, locally contaminated components, etc. If any of the above anomalies is identified as present, the obtained component anomaly identification result is anomaly; if none of the above anomaly identifications are present, the obtained component anomaly identification result is not anomaly.

[0049] In a specific embodiment, step S130 includes the following sub-steps: calculating the mean value of the region pixels corresponding to each component region in the region pixel feature information of the image feature information; determining whether the mean value of the region pixels of each component region is within the corresponding mean value range according to the mean value range of each component region in the component anomaly identification rule, and obtaining first anomaly judgment information; if the first anomaly judgment result of the image feature information is yes, determining whether the pixel difference between the pixel value of the pixel point in each component region and the mean value of the region pixels is within the corresponding difference value range according to the difference value range of each component region in the component anomaly identification rule, and obtaining second anomaly judgment result; if the second anomaly judgment result of the image feature information is yes, obtaining a non-abnormal anomaly recognition result; if the first anomaly judgment result of the image feature information is no or the second anomaly judgment result is no, obtaining an abnormal anomaly recognition result.

[0050] Specifically, the mean pixel value of each component region can be obtained from the region pixel feature information. The mean pixel value is the average pixel value of all pixels within a component region. Further, it is determined whether the mean pixel value of a component region falls within the range of the corresponding mean pixel values ​​for that component region. This helps determine if any components in each component region are missing. If the mean pixel value is not within the corresponding mean pixel value range, it indicates that a component in that component region is missing; if it is within the corresponding mean pixel value range, it indicates that no component in that component region is missing. By determining whether the mean pixel value of each component region falls within the corresponding mean pixel value range, the first anomaly detection information can be obtained.

[0051] Furthermore, the pixel difference between the pixel value of each pixel in the component region and the average pixel value of the component region can be calculated, and it can be determined whether the pixel difference of each pixel is within the corresponding difference value range of the component region. If the pixel difference of a pixel is not within the corresponding difference value range, it indicates that the pixel value of that pixel is abnormal, that is, there is an abnormality of local film damage or local dirt in the component region. By judging the pixel values ​​of each pixel in each component region of an image feature information using the above method, the second anomaly judgment result of the image feature information is obtained.

[0052] If both the first and second anomaly detection results for the chip image are yes, then a non-abnormal anomaly detection result is obtained. If either the first or second anomaly detection result for the chip image is no, then an abnormal anomaly detection result is obtained.

[0053] S140. If the component anomaly identification result of the image feature information is not abnormal, perform integrity identification on the image feature information according to the preset integrity identification rules to obtain the integrity identification result of whether it is complete.

[0054] If the component anomaly identification result of the image feature information is not abnormal, the image feature information is subjected to integrity identification according to the preset integrity identification rules to obtain a completeness identification result. If the component anomaly identification result of the image feature information is not abnormal, the integrity identification of the image feature information continues according to the integrity identification rules; wherein, integrity identification is to identify whether the outline of the device is complete and whether the outline of the device is offset. If the outline of the device is incomplete or offset, the obtained integrity identification result is incomplete.

[0055] In a specific embodiment, step S140 includes the following sub-steps: determining whether the maximum curvature of the contour lines of each component contour region in the feature pixel contour information of the image feature information is located within the curvature interval corresponding to each component contour region in the integrity recognition rule, and obtaining a first integrity judgment result; if the first integrity judgment result is yes, calculating the angle value between the contour lines of each component contour region in the feature pixel contour information and the image border of the corresponding chip image; determining whether the angle value of the feature pixel contour information is not greater than the angle threshold in the integrity recognition rule, and obtaining a second integrity judgment result; if the second integrity judgment result of the image feature information is yes, obtaining a complete integrity recognition result; if the first integrity judgment result or the second integrity judgment result of the image feature information is no, obtaining an incomplete integrity recognition result.

[0056] Specifically, the maximum curvature of each contour line in the feature pixel contour information of each component in the image feature information can be calculated. The curvature of each point on the contour line can be calculated, and the maximum value of the curvature is taken as the maximum curvature of the contour line. It is determined whether the maximum curvature of each contour line in the component contour area is located within the curvature interval corresponding to the component contour area. If the maximum curvature of each contour line in the component contour area is located within the curvature interval, it indicates that the device in the component contour area is intact; if the maximum curvature of a certain contour line is not within the curvature interval, it indicates that the edge of the device in the component contour area is locally damaged, and the device in the component contour area is incomplete. The above judgment is performed on each component contour area in the image feature information to obtain the first integrity judgment result.

[0057] Furthermore, the angle between the line connecting the two ends of the contour line of the component contour region and the corresponding image border is calculated; that is, the angle between the contour line and the nearest image border is calculated. For example, for the left contour line, the angle between the left contour line and the left border of the image is calculated, and for the lower contour line, the angle between the lower contour line and the lower border of the image is calculated. It is then determined whether the angle value of each contour line in the feature pixel contour information is not greater than the angle threshold set in the integrity recognition rule, thus obtaining the second integrity judgment result.

[0058] If both the first and second integrity judgment results of the chip image are yes, a complete integrity recognition result is obtained. If either the first or second integrity judgment result of the chip image is no, an incomplete integrity recognition result is obtained.

[0059] S150. If the integrity recognition result of the image feature information is complete, perform wiring recognition on the image feature information according to the preset wiring recognition rules to obtain the wiring recognition result of whether the wiring is abnormal.

[0060] If the integrity recognition result of the image feature information is complete, the image feature information is subjected to wiring identification according to the preset wiring identification rules to obtain a wiring identification result indicating whether the wiring is abnormal. Furthermore, wiring identification can be performed on the image feature information according to the wiring identification rules, that is, to determine whether there is a break in the connection between ports or whether the connecting wires are connected incorrectly.

[0061] Specifically, the end positions of the connecting lines corresponding to each connecting line contour can be obtained from the feature pixel contour information of the image feature information. These end positions are then matched with the port connection information in the wiring identification rules. If the end positions of the connecting line contours in the image feature information match the port connection information, a normal wiring identification result is obtained. If the end positions of the connecting line contours in the image feature information do not match the port connection information, an abnormal wiring identification result is obtained. Mismatches include situations where two ports with a connection relationship in the port connection information do not match one end position of any connecting line contour, or do not match both end positions of any connecting line contour, i.e., a broken connection or misaligned connection.

[0062] S160. Determine the abnormal chip image and the corresponding abnormal type based on the component abnormality identification result, the integrity identification result, and the wiring identification result.

[0063] The abnormal chip image and its corresponding abnormal type are determined based on the component anomaly identification result, the integrity identification result, and the wiring identification result. Based on these results, chip images with an abnormal component identification result, an incomplete integrity identification result, or an abnormal wiring identification result can be obtained and identified as abnormal chip images. The abnormal type of the abnormal chip image can then be determined based on its identification result. For example, if the component anomaly identification result of the abnormal chip image is abnormal, the abnormal type is component anomaly; if the integrity identification result is incomplete, the abnormal type is component incomplete; if the wiring identification result is abnormal, the abnormal type is wiring anomaly.

[0064] S170. Combine the abnormal coordinates of each abnormal chip image in the optical image with the corresponding abnormal type to obtain the abnormal detection information corresponding to the optical image.

[0065] The abnormal coordinates of each abnormal chip image in the optical image are combined with its corresponding abnormal type to obtain the abnormal detection information corresponding to the optical image. The abnormal coordinates of the abnormal chip image in the optical image can be determined based on its row and column numbers. Combining the abnormal coordinates of each abnormal chip image with its abnormal type yields the abnormal detection information corresponding to the optical image.

[0066] In a specific embodiment, after step S170, the method further includes the steps of: generating a corresponding abnormal icon based on the abnormal type of the abnormal chip image; and generating an abnormal detection point map corresponding to the optical image based on the abnormal icon and the abnormal coordinate position of each of the abnormal chip images.

[0067] Furthermore, to improve the convenience for testing personnel in obtaining testing information, corresponding anomaly icons can be generated based on the anomaly type of the abnormal chip image. Different anomaly types correspond to different colors and / or different shapes of anomaly icons. For example, the anomaly icon corresponding to an abnormal chip image with component anomaly is red, the anomaly icon corresponding to an incomplete component is yellow, and the anomaly icon corresponding to wiring anomaly is purple.

[0068] Based on the anomaly icons and the anomaly coordinates of the anomaly chip images, an anomaly detection point map corresponding to the optical image can be generated. This anomaly detection point map contains an icon for each sensor chip, and the arrangement of the sensor icons in the anomaly detection point information corresponds to the arrangement of the sensor chips on the circuit board. Sensor chips without anomalies have white icons, while those with anomalies display their corresponding anomaly icons at the appropriate positions in the anomaly detection point map.

[0069] The image recognition-based intelligent detection method disclosed in the above embodiments includes: segmenting an optical image acquired by a camera device to obtain a chip image for each chip; extracting image feature information from each chip image and performing anomaly identification, integrity identification, and wiring identification; determining the anomaly type of the abnormal chip image based on the anomaly identification results, integrity identification results, and wiring identification results; and combining the anomaly coordinate position and anomaly type of the abnormal chip image to obtain anomaly detection information. This method can segment an optical image to obtain a chip image for each chip, identify each chip image individually to determine whether it is an abnormal chip image, and combine the anomaly coordinate position and anomaly type of the abnormal chip image to obtain anomaly detection information, significantly improving the efficiency and reliability of detecting the assembly quality of sensor chips.

[0070] This invention also provides an image recognition-based intelligent detection device, which can be configured in a detection device and is used to execute any of the aforementioned image recognition-based intelligent detection methods. Specifically, please refer to... Figure 3 , Figure 3 This is a schematic block diagram of an image recognition-based intelligent detection device provided in an embodiment of the present invention.

[0071] like Figure 3 As shown, the intelligent detection device 100 based on image recognition includes an image segmentation unit 110, an image feature information acquisition unit 120, a first recognition unit 130, a second recognition unit 140, a third recognition unit 150, a determination unit 160, and a combination unit 170.

[0072] The image segmentation unit 110 is used to segment the optical image according to a preset image segmentation rule if it receives the optical image acquired by the camera device, so as to obtain the chip image corresponding to each chip in the optical image;

[0073] The image feature information acquisition unit 120 is used to extract the corresponding image feature information from each chip image according to a preset image feature extraction model.

[0074] The first identification unit 130 is used to perform anomaly identification on each of the image feature information according to the preset component anomaly identification rules, and obtain the component anomaly identification result of whether it is abnormal.

[0075] The second identification unit 140 is used to perform integrity identification on the image feature information according to a preset integrity identification rule if the component abnormality identification result of the image feature information is not abnormal, and obtain an integrity identification result of whether it is complete.

[0076] The third identification unit 150 is used to identify the wiring of the image feature information according to the preset wiring identification rules if the integrity identification result of the image feature information is complete, and to obtain the wiring identification result of whether the wiring is abnormal.

[0077] The determining unit 160 is used to determine the abnormal chip image and the corresponding abnormal type based on the component abnormality identification result, the integrity identification result and the wiring identification result;

[0078] The combination unit 170 is used to combine the abnormal chip images of each abnormal chip with the abnormal coordinate position of the optical image and the corresponding abnormal type to obtain the abnormal detection information corresponding to the optical image.

[0079] The image recognition-based intelligent detection device provided in this embodiment of the invention applies the aforementioned image recognition-based intelligent detection method. It segments the optical image acquired by the camera device to obtain chip images for each chip. Image feature information is extracted from each chip image, and anomaly identification, integrity identification, and wiring identification are performed. Based on the anomaly identification results, integrity identification results, and wiring identification results, the anomaly type of the abnormal chip image is determined. The anomaly coordinate position and anomaly type of the abnormal chip image are combined to obtain anomaly detection information. This method can segment the optical image to obtain chip images for each chip, and identify each chip image individually to determine whether it is an abnormal chip image. By combining the anomaly coordinate position and anomaly type of the abnormal chip image to obtain anomaly detection information, it significantly improves the efficiency and reliability of detecting the assembly quality of sensor chips.

[0080] The aforementioned image recognition-based intelligent detection device can be implemented as a computer program, which can, for example... Figure 4 It runs on the computer device shown.

[0081] Please see Figure 4 , Figure 4 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. The computer device can be a detection device for performing an image recognition-based intelligent detection method to intelligently detect the assembly quality of sensor chips.

[0082] See Figure 4 The computer device 500 includes a processor 502, a memory, and a communication interface 505 connected via a communication bus 501. The memory may include a storage medium 503 and internal memory 504.

[0083] The storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, it enables the processor 502 to perform an intelligent detection method based on image recognition. The storage medium 503 may be a volatile storage medium or a non-volatile storage medium.

[0084] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0085] The internal memory 504 provides an environment for the operation of the computer program 5032 in the storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute an intelligent detection method based on image recognition.

[0086] This communication interface 505 is used for network communication, such as providing data transmission. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device 500 to which the present invention is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0087] The processor 502 is used to run the computer program 5032 stored in the memory to implement the corresponding functions in the image recognition-based intelligent detection method described above.

[0088] Those skilled in the art will understand that Figure 4The embodiments of the computer device shown do not constitute a limitation on the specific configuration of the computer device. In other embodiments, the computer device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. For example, in some embodiments, the computer device may include only memory and a processor. In such embodiments, the structure and function of the memory and processor are different from those shown. Figure 4 The embodiments shown are consistent and will not be described again here.

[0089] It should be understood that, in this embodiment of the invention, the processor 502 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0090] In another embodiment of the invention, a computer-readable storage medium is provided. This computer-readable storage medium may be volatile or non-volatile. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps included in the image recognition-based intelligent detection method described above.

[0091] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0092] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Units with the same function may be grouped into one unit. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, or it may be an electrical, mechanical, or other form of connection.

[0093] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0094] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0095] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned computer-readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks.

[0096] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An intelligent detection method based on image recognition, characterized in that, The method is applied in a detection device, which is connected to a camera device via a network to transmit data information. The method includes: If an optical image acquired by the camera device is received, the optical image is segmented according to a preset image segmentation rule to obtain a chip image corresponding to each chip in the optical image; The corresponding image feature information is extracted from each chip image according to the preset image feature extraction model; According to preset component anomaly identification rules, anomaly identification is performed on each of the image feature information to obtain the component anomaly identification result. The component anomaly identification involves identifying whether there are missing components, local film damage, and local contamination in the chip image, including: Calculate the mean value of the region pixels corresponding to each component region in the region pixel feature information of the image feature information; Based on the range of average pixel values ​​corresponding to each component region in the component anomaly identification rules, determine whether the average pixel value of each component region is within the corresponding range of average pixel values, and obtain the first anomaly identification information. If the first anomaly judgment result of the image feature information is yes, according to the difference value range corresponding to each component region in the component anomaly recognition rule, it is determined whether the pixel difference between the pixel value of the pixel point in each component region and the average pixel value of the region are all within the corresponding difference value range, and the second anomaly judgment result is obtained. If the second anomaly judgment result of the image feature information is yes, an anomaly recognition result that is not abnormal is obtained; If the first anomaly judgment result or the second anomaly judgment result of the image feature information is negative, an anomaly identification result is obtained. If the component anomaly identification result of the image feature information is not abnormal, the image feature information is subjected to integrity identification according to the preset integrity identification rules to obtain the integrity identification result of whether it is complete; the integrity identification is to identify whether the outline of the component is complete and whether it is offset. If the integrity recognition result of the image feature information is complete, the image feature information is subjected to wiring recognition according to the preset wiring recognition rules to obtain the wiring recognition result of whether the wiring is abnormal; The abnormal chip image and the corresponding abnormal type are determined based on the component anomaly identification result, the integrity identification result, and the wiring identification result. The abnormal chip images are combined with the abnormal coordinates of the optical image and the corresponding abnormal type to obtain the abnormal detection information corresponding to the optical image.

2. The intelligent detection method based on image recognition according to claim 1, characterized in that, The step of segmenting the optical image according to a preset image segmentation rule to obtain a chip image corresponding to each chip in the optical image includes: The corresponding template boundary position and template boundary size are determined based on the positioning marks on the optical image; The default cutting template in the image segmentation rule is adjusted according to the template boundary size to obtain a target cutting template that matches the template boundary size; The target cutting template is placed on top of the optical image according to the template boundary position; The optical image is cut according to the target cutting template to obtain a chip image.

3. The intelligent detection method based on image recognition according to claim 2, characterized in that, After obtaining the chip image by cutting the optical image according to the target cutting template, the process further includes: Obtain the corresponding QR code image from the QR code image cutting area set in the target cutting template; The QR code image is identified using a preset QR code recognition model to obtain QR code information corresponding to the optical image.

4. The intelligent detection method based on image recognition according to claim 1, characterized in that, The step of extracting corresponding image feature information from each chip image according to a preset image feature extraction model includes: Based on the feature extraction region in the image feature extraction model, region pixel features are extracted from each of the chip images to obtain the region pixel feature information corresponding to each of the chip images; Based on the feature pixel range in the image feature extraction model, feature pixel contours are extracted from each of the chip images to obtain feature pixel contour information corresponding to each of the chip images; The region pixel feature information and the feature pixel contour information are combined to form the corresponding image feature information.

5. The intelligent detection method based on image recognition according to claim 1, characterized in that, The step of performing integrity identification on the image feature information according to preset integrity identification rules to obtain an integrity identification result of whether the image is complete includes: Determine whether the maximum curvature of the contour lines of each component contour region in the feature pixel contour information of the image feature information is located within the curvature interval corresponding to each component contour region in the integrity recognition rule, and obtain the first integrity judgment result; If the first integrity judgment result is yes, calculate the angle between the contour line of each component contour region in the feature pixel contour information and the image border of the corresponding chip image. Determine whether the included angle values ​​of the feature pixel contour information are all not greater than the included angle threshold in the integrity recognition rule to obtain a second integrity judgment result; If the second integrity judgment result of the image feature information is yes, a complete integrity recognition result is obtained; If the first integrity judgment result or the second integrity judgment result of the image feature information is negative, an incomplete integrity recognition result is obtained.

6. The intelligent detection method based on image recognition according to claim 1, characterized in that, After combining the abnormal chip images of each abnormal chip with the corresponding abnormal type at the abnormal coordinate position of the optical image to obtain the abnormal detection information corresponding to the optical image, the method further includes: Generate a corresponding anomaly icon based on the anomaly type of the abnormal chip image; An anomaly detection point map corresponding to the optical image is generated based on the anomaly icon and the anomaly coordinate positions of each anomaly chip image.

7. An intelligent detection device based on image recognition, characterized in that, The image recognition-based intelligent detection device is used to execute the image recognition-based intelligent detection method as described in any one of claims 1-6. The device is configured in a detection device, which is network-connected to a camera device to transmit data information. The device includes: An image segmentation unit is used to segment an optical image acquired by the camera device according to a preset image segmentation rule to obtain a chip image corresponding to each chip in the optical image if the optical image is received. The image feature information acquisition unit is used to extract the corresponding image feature information from each chip image according to the preset image feature extraction model; The first identification unit is used to perform anomaly identification on each of the image feature information according to the preset component anomaly identification rules, and obtain the component anomaly identification result of whether it is abnormal. The second identification unit is used to perform integrity identification on the image feature information according to a preset integrity identification rule if the component anomaly identification result of the image feature information is not abnormal, and obtain an integrity identification result of whether it is complete. The third identification unit is used to identify the wiring of the image feature information according to the preset wiring identification rules if the integrity identification result of the image feature information is complete, and to obtain the wiring identification result of whether the wiring is abnormal. The determining unit is used to determine the abnormal chip image and the corresponding abnormal type based on the component abnormality identification result, the integrity identification result and the wiring identification result; The combination unit is used to combine the abnormal chip images of each abnormal chip with the abnormal coordinate position of the optical image and the corresponding abnormal type to obtain the abnormal detection information corresponding to the optical image.

8. A computer device, characterized in that, The device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in a memory, it implements the steps of the intelligent detection method based on image recognition as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the intelligent detection method based on image recognition as described in any one of claims 1-6.

Images