Intelligent detection method and device based on image recognition, equipment and medium

Abstract

The invention discloses an intelligent detection method, device and equipment based on image recognition and a medium, and the method comprises the steps: separating an optical image collected by camera equipment to obtain a chip image of each chip, extracting image feature information from each chip image, and carrying out the abnormality recognition, integrity recognition and wiring recognition; and determining the abnormal type of the abnormal chip image according to the abnormal identification result, the integrity identification result and the wiring identification result, and combining the abnormal coordinate position of the abnormal chip image with the abnormal type to obtain abnormal detection information. According to the method, the optical image can be separated to obtain the chip image of each chip, each chip image is independently identified to judge whether the chip image is the abnormal chip image, and the abnormal coordinate position and the abnormal type of the abnormal chip image are synthesized to obtain the abnormal detection information. And the efficiency and the reliability of detecting the assembly quality of the sensor chip are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent detection, and in particular, to an intelligent detection method, device, equipment and medium based on image recognition. Background Art

[0002] During the production process of sensors, chips need to be assembled at preset positions on large-sized circuit boards, and the chips need to be electrically connected to other components. To detect the assembly quality of the chips, it is necessary to manually check whether the sensor chips assembled on the entire circuit board are abnormal, and cut off the sensors at the abnormal positions after cutting the sensors from the entire circuit board. However, in the prior art, by irradiating the circuit board with a magnifying glass and manually checking, it is impossible to quickly detect the assembly quality of sensor chips according to a unified standard, which affects the efficiency and accuracy of sensor chip detection. Therefore, there is a problem in the prior art method that it is impossible to quickly detect the assembly quality of sensor chips on the circuit board. Summary of the Invention

[0003] Embodiments of the present invention provide an intelligent detection method, device, equipment and medium based on image recognition, aiming to solve the problem in the prior art method that it is impossible to quickly detect the assembly quality of sensor chips on the circuit board.

[0004] In a first aspect, embodiments of the present invention provide an intelligent detection method based on image recognition. The method is applied to a detection device, and the detection device is network-connected to a camera device to achieve data information transmission. Wherein, the method includes:

[0005] If an optical image collected by the camera device is received, segment the optical image according to a preset image segmentation rule to obtain a chip image corresponding to each chip in the optical image;

[0006] Extract corresponding image feature information from each chip image according to a preset image feature extraction model;

[0007] Perform abnormality recognition on each of the image feature information according to a preset component abnormality recognition rule to obtain a component abnormality recognition result of whether there is an abnormality;

[0008] If the component abnormality recognition result of the image feature information is not abnormal, perform integrity recognition on the image feature information according to a preset integrity recognition rule to obtain an integrity recognition result of whether it is complete;

[0009] If the integrity recognition result of the image feature information is complete, perform wiring recognition on the image feature information according to a preset wiring recognition rule to obtain a wiring recognition result of whether the wiring is abnormal;

[0010] Determine the abnormal chip image and the corresponding abnormal type according to the component abnormality recognition result, the integrity recognition result, and the wiring recognition result;

[0011] Combine the abnormal coordinate positions of each abnormal chip image in the optical image with the corresponding abnormal type to obtain the abnormal detection information corresponding to the optical image.

[0012] In a second aspect, an embodiment of the present invention further provides an intelligent detection device based on image recognition. The device is used to execute the intelligent detection method based on image recognition as described in the first aspect above. The device is configured in a detection device, and the detection device is network-connected to a camera device to achieve data information transmission. The device includes:

[0013] An image segmentation unit, which is used to segment the optical image according to a preset image segmentation rule to obtain a chip image corresponding to each chip in the optical image if the optical image collected by the camera device is received;

[0014] An image feature information acquisition unit, which is used to extract corresponding image feature information from each chip image according to a preset image feature extraction model;

[0015] A first recognition unit, which is used to perform abnormal recognition on each of the image feature information according to a preset component abnormality recognition rule to obtain a component abnormality recognition result of whether there is an abnormality;

[0016] A second recognition unit, which is used to perform integrity recognition on the image feature information according to a preset integrity recognition rule to obtain an integrity recognition result of whether it is complete if the component abnormality recognition result of the image feature information is not abnormal;

[0017] A third recognition unit, which is used to perform wiring recognition on the image feature information according to a preset wiring recognition rule to obtain a wiring recognition result of whether the wiring is abnormal if the integrity recognition result of the image feature information is complete;

[0018] A determination unit, which is used to determine the abnormal chip image and the corresponding abnormal type according to the component abnormality recognition result, the integrity recognition result, and the wiring recognition result;

[0019] A combination unit, which is used to combine the abnormal coordinate positions of each abnormal chip image in the optical image with the corresponding abnormal type to obtain the abnormal detection information corresponding to the optical image.

[0020] In a third aspect, an embodiment of the present invention further provides a computer device, where the device includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0021] The memory is used to store a computer program;

[0022] The processor is configured to implement the steps of the intelligent detection method based on image recognition described in the first aspect above when executing the program stored on the memory.

[0023] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the intelligent detection method based on image recognition described in the first aspect above.

[0024] An embodiment of the present invention provides an intelligent detection method, device, equipment, and medium based on image recognition. The method includes: separating the optical image collected by the imaging device to obtain the chip image of each chip, extracting image feature information from each chip image and performing anomaly recognition, integrity recognition, and wiring recognition, determining the anomaly type of the abnormal chip image according to the anomaly recognition result, integrity recognition result, and wiring recognition result, and combining the abnormal coordinate position and anomaly type of the abnormal chip image to obtain anomaly detection information. The above method can separate the optical image to obtain the chip image of each chip, and separately identify each chip image to determine whether the chip image is an abnormal chip image, and combine the abnormal coordinate position and anomaly type of the abnormal chip image to obtain anomaly detection information, greatly improving the efficiency and reliability of detecting the assembly quality of sensor chips. Description of the Drawings

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1 It is a flowchart of the intelligent detection method based on image recognition provided by an embodiment of the present invention;

[0027] Figure 2 It is a schematic diagram of the application scenario of the intelligent detection method based on image recognition provided by an embodiment of the present invention;

[0028] Figure 3 It is a schematic block diagram of the intelligent detection device based on image recognition provided by an embodiment of the present invention;

[0029] Figure 4 It is a schematic block diagram of a computer device provided by an embodiment of the present invention. Specific implementation manners

[0030] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0031] It should be understood that when used in this specification and the appended claims, the terms "comprises" and "comprising" indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0032] It should also be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0033] It should be further understood that the term "and / or" used in the specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0034] Please refer to Figure 1 , as shown in the figure, an embodiment of the present invention application provides an intelligent detection method based on image recognition. This method is applied to the detection device 10 and is executed by the application software installed in the detection device 10. The detection device 10 is network-connected to the imaging device 20 to achieve the transmission of data information. Among them, the imaging device 20 is also a device used to magnify the circuit board and collect the circuit board image, and the imaging device 20 can be used to collect the optical image of the circuit board. The detection device 10 is also a terminal device used to detect and identify the received optical image, such as a desktop computer, a laptop computer, a tablet computer, or a mobile phone, etc. As Figure 1 shown, this method includes steps S110 to S170.

[0035] S110. If the optical image collected by the imaging device is received, segment the optical image according to the preset image segmentation rule to obtain the chip image corresponding to each chip in the optical image.

[0036] If the optical image collected by the imaging device is received, the optical image is segmented according to a preset image segmentation rule to obtain a chip image corresponding to each chip in the optical image. When the imaging device collects the optical image of the circuit board, the optical image can be sent to the detection device. Since there are multiple sensor chips on the circuit board at this time, it is necessary to segment the optical image through the image segmentation rule to obtain the chip image corresponding to each chip, and perform separate recognition and detection processing on each chip image.

[0037] In a specific embodiment, step S110 includes sub-steps: determining the corresponding template boundary position and template boundary size according to the positioning marks on the optical image; adjusting the default cutting template in the image segmentation rule according to the template boundary size to obtain a target cutting template matching the template boundary size; covering the target cutting template on the upper layer of the optical image according to the template boundary position; and cutting the optical image according to the target cutting template to obtain a chip image.

[0038] Specifically, positioning marks are posted / engraved on the circuit board, so the optical image also contains positioning marks. The optical image may contain multiple positioning marks. For example, positioning marks are set at the four top corners of the circuit board, or positioning marks are set at two diagonal corners of the circuit board. Since the positioning marks are set at fixed positions on the circuit board, the position of the outer frame formed by enclosing the positioning marks can be determined as the template boundary position through the distance between the positioning marks and the position of the positioning marks, and the boundary size of the outer frame calculated according to the distance between the positioning marks is used as the template boundary size.

[0039] Further, the default cutting template is scaled and adjusted according to the template boundary size, so that the obtained target cutting template after adjustment matches the template boundary size. The target cutting template is covered on the upper layer of the optical image according to the template boundary position and the optical image is cut. The target cutting template contains multiple warp lines and weft lines, and the optical image can be cut through the warp lines and weft lines to obtain the chip images corresponding to each chip.

[0040] In a specific embodiment, after cutting the optical image according to the target cutting template to obtain a chip image, it further includes: obtaining a corresponding two-dimensional code image from the set two-dimensional code image cutting area in the target cutting template; and identifying the two-dimensional code image according to a preset two-dimensional code recognition model to obtain two-dimensional code information corresponding to the optical image.

[0041] Specifically, a two-dimensional code image cutting area is also set in the target cutting template. The two-dimensional code image corresponding to the two-dimensional code image cutting area can be obtained, and the two-dimensional code image is recognized by a two-dimensional code recognition model. Among them, the two-dimensional code image can be a bar code or a two-dimensional code in the form of scatter points. Then, the corresponding recognition templates in the two-dimensional code recognition model can be used to perform corresponding recognition on different types of two-dimensional code images, so as to obtain two-dimensional code information, which is used to uniquely identify the currently detected circuit board.

[0042] S120. Extract the corresponding image feature information from each chip image according to the pre-set image feature extraction model.

[0043] Extract the corresponding image feature information from each chip image according to the pre-set image feature extraction model. The image feature extraction model is the specific model used for image feature extraction, and the corresponding image feature information can be extracted from the chip image.

[0044] In a specific embodiment, step S120 includes sub-steps: extracting regional pixel features from each of the chip images according to the feature extraction regions in the image feature extraction model to obtain regional pixel feature information corresponding to each of the chip images; extracting feature pixel contours from each of the chip images according to the feature pixel intervals in the image feature extraction model to obtain feature pixel contour information corresponding to each of the chip images; combining the regional pixel feature information and the feature pixel contour information into the corresponding image feature information.

[0045] The image feature extraction model includes multiple feature extraction regions, and each feature extraction region corresponds to a specific region where a component is located. The pixel values of the pixel points included in the region can be extracted from each chip image according to the extraction position of the feature extraction region as the regional pixel features of the feature extraction region. Then, the regional pixel features of each feature extraction region in the chip image are combined into the regional pixel feature information of the chip image. Further, the image feature extraction model also includes multiple feature pixel intervals, and each feature pixel interval is used to define a pixel value range. Then, it can be judged whether the pixel values of the pixel points in the chip image are within the feature pixel interval, so as to obtain all the pixel points located in a feature pixel interval and extract the outer contour of the region composed of all the pixel points, so as to obtain the feature pixel contour of the feature pixel interval. Obtaining the feature pixel contour information of each feature pixel interval in the chip image is combined into the feature pixel contour information of the chip image.

[0046] Combining the regional pixel feature information and the feature pixel contour information of the chip image can obtain the image feature information of the chip image.

[0047] S130. Perform anomaly recognition on each of the image feature information according to a preset component anomaly recognition rule to obtain a component anomaly recognition result indicating whether there is an anomaly.

[0048] Perform anomaly recognition on each of the image feature information according to a preset component anomaly recognition rule to obtain a component anomaly recognition result indicating whether there is an anomaly. Further, the component anomaly recognition rule can be used to recognize whether there is an anomaly in the components in the chip image. Specifically, the anomaly recognition rule can be used to perform anomaly recognition on the image feature information of each chip image respectively. Anomaly recognition is to recognize whether there are anomalies such as missing MEMS components, missing ASIC components, local film breakage of components, and local contamination of components in the chip image. If any of the above anomalies is judged to be yes, the obtained component anomaly recognition result is abnormal; if all of the above anomaly judgments are no, the obtained component anomaly recognition result is not abnormal.

[0049] In a specific embodiment, step S130 includes sub-steps: calculating the regional pixel mean value corresponding to each component region in the regional pixel feature information of the image feature information; according to the pixel mean value range corresponding to each component region in the component anomaly recognition rule, determining whether the regional pixel mean values of each component region are all within the corresponding pixel mean value range to obtain a first anomaly judgment information; if the first anomaly judgment result of the image feature information is yes, according to the difference value range corresponding to each component region in the component anomaly recognition rule, determining whether the pixel difference values between the pixel values of the pixel points in each component region and the regional pixel mean value are all within the corresponding difference value range to obtain a second anomaly judgment result; if the second anomaly judgment result of the image feature information is yes, obtain an abnormal recognition result indicating no anomaly; if the first anomaly judgment result of the image feature information is no or the second anomaly judgment result is no, obtain an abnormal recognition result indicating an anomaly.

[0050] Specifically, the regional pixel mean value of each component region can be obtained from the regional pixel feature information. The regional pixel mean value is also the average value of the pixel values of all pixel points within a component region. Further, determine whether the regional pixel mean value of the component region is within the pixel mean value range corresponding to the component region, so as to determine whether there is a missing component in each component region. If the regional pixel mean value is not within the corresponding pixel mean value range, it indicates that there is a missing component in the component region; if the regional pixel mean value is within the corresponding pixel mean value range, it indicates that there is no missing component in the component region. By determining whether the regional pixel mean values of each component region are all within the corresponding pixel mean value range, the first anomaly judgment information can be obtained.

[0051] Further, the pixel difference between the pixel value of a pixel in the component area and the regional pixel average value of the component area can be calculated, and it can be determined whether the pixel difference of each pixel is within the difference value range corresponding to the component area. If the pixel difference of a pixel is not within the corresponding difference value range, it indicates that the pixel value of this pixel is abnormal, that is, there is an abnormality such as local film breakage or local contamination in the component area. By using the above method to judge the pixel values of the pixels in each component area of an image feature information respectively, the second abnormality judgment result of the image feature information can be obtained.

[0052] If the first abnormality judgment result of the chip image is yes and the second abnormality judgment result is yes, an abnormality recognition result of not abnormal can be obtained. If the first abnormality judgment result of the chip image is no or the second abnormality judgment result is no, an abnormality recognition result of abnormal can be obtained.

[0053] S140. If the component abnormality recognition result of the image feature information is not abnormal, perform integrity recognition on the image feature information according to the preset integrity recognition rules to obtain an integrity recognition result of whether it is complete.

[0054] If the component abnormality recognition result of the image feature information is not abnormal, perform integrity recognition on the image feature information according to the preset integrity recognition rules to obtain an integrity recognition result of whether it is complete. If the component abnormality recognition result of the image feature information is not abnormal, continue to perform integrity recognition on the image feature information according to the integrity recognition rules; among them, integrity recognition is to recognize whether the contour line of the device is complete and whether the contour line of the device is offset. If the device contour line is incomplete or the device contour line is offset, the obtained integrity recognition result is incomplete.

[0055] In a specific embodiment, step S140 includes sub-steps: determining whether the maximum curvature of the contour line of each component contour area in the feature pixel contour information of the image feature information is within the curvature interval corresponding to each component contour area in the integrity recognition rules to obtain a first integrity judgment result; if the first integrity judgment result is yes, calculate the included angle value between the contour line of each component contour area in the feature pixel contour information and the image border of the corresponding chip image; determine whether the included angle value of the feature pixel contour information is not greater than the included angle threshold in the integrity recognition rules to obtain a second integrity judgment result; if the second integrity judgment result of the image feature information is yes, obtain a complete integrity recognition result; if the first integrity judgment result of the image feature information is no or the second integrity judgment result is no, obtain an incomplete integrity recognition result.

[0056] Specifically, the maximum curvature of each contour line in the contour region of each component in the feature pixel contour information capable of calculating image feature information can be calculated. The curvature of each point on the contour line can be calculated, and the maximum value of the curvatures is taken as the maximum curvature of the contour line. It is determined whether the maximum curvatures of the contour lines in the component contour region are all within the corresponding curvature intervals of the component contour regions. If the maximum curvatures of the contour lines in the component contour region are all within the curvature intervals, it indicates that the device in the component contour region is complete; if the maximum curvature of a certain contour line is not within the curvature interval, it indicates that the edge of the device in the component contour region is partially damaged and the device in the component contour region is incomplete. The above judgments are respectively performed on each component contour region in the image feature information, thereby obtaining the first integrity judgment result.

[0057] Further, the included angle value formed by the line connecting the two ends of the contour line of the component contour region and the corresponding image border is calculated, that is, the included angle value between the contour line and the nearest image border is calculated; for example, for the left contour line, the included angle value between the left contour line and the left image border is calculated, and for the lower contour line, the included angle value between the lower contour line and the lower image border is calculated. It is determined whether the included angle values of each contour line in the feature pixel contour information are all not greater than the included angle threshold set in the integrity recognition rule, thereby obtaining the second integrity judgment result.

[0058] If the first integrity judgment result of the chip image is yes and the second integrity judgment result is yes, a complete integrity recognition result is obtained. If the first integrity judgment result of the chip image is no or the second integrity judgment result is no, an incomplete integrity recognition result is obtained.

[0059] S150. If the integrity recognition result of the image feature information is complete, the image feature information is subjected to wiring recognition according to the preset wiring recognition rule, and a wiring recognition result indicating whether the wiring is abnormal is obtained.

[0060] If the integrity recognition result of the image feature information is complete, the image feature information is subjected to wiring recognition according to the preset wiring recognition rule, and a wiring recognition result indicating whether the wiring is abnormal is obtained. Further, the image feature information can be subjected to wiring recognition according to the wiring recognition rule, that is, it is determined whether there is a disconnection between ports and whether the connection wires are connected wrongly.

[0061] Specifically, the end positions of the connecting lines corresponding to the connecting line contours can be obtained from the connecting line contours in the feature pixel contour information of the image feature information, and the end positions of the connecting line contours are matched and judged with the port connection information in the wiring recognition rule. If the end position of the connecting line contour in the image feature information matches the port connection information, a normal wiring recognition result is obtained. If the end position of the connecting line contour in the image feature information does not match the port connection information, an abnormal wiring recognition result is obtained. The situation of non - matching includes that two ports with a connection relationship in the port connection information do not match one end position of any connecting line contour, or do not match two end positions of any connecting line contour, that is, disconnection or connection misalignment.

[0062] S160. Determine the abnormal chip image and the corresponding abnormal type according to the component abnormal recognition result, the integrity recognition result, and the wiring recognition result.

[0063] Determine the abnormal chip image and the corresponding abnormal type according to the component abnormal recognition result, the integrity recognition result, and the wiring recognition result. According to the component abnormal recognition result, the integrity recognition result, and the wiring recognition result, a chip image with an abnormal component abnormal recognition result, an incomplete integrity recognition result, or an abnormal wiring recognition result can be obtained and determined as an abnormal chip image. According to the recognition result of the abnormal chip image, the abnormal type of the abnormal chip image can be determined. For example, if the component abnormal recognition result of the abnormal chip image is abnormal, the abnormal type is component abnormality; if the integrity recognition result of the abnormal chip image is incomplete, the abnormal type is component incompleteness; if the wiring recognition result of the abnormal chip image is abnormal, the abnormal type is wiring abnormality.

[0064] S170. Combine the abnormal coordinate positions of each abnormal chip image in the optical image with the corresponding abnormal type to obtain the abnormal detection information corresponding to the optical image.

[0065] Combine the abnormal coordinate positions of each abnormal chip image in the optical image with the corresponding abnormal type to obtain the abnormal detection information corresponding to the optical image. According to the row number and column number of the abnormal chip image, the abnormal coordinate position of the abnormal chip image in the optical image can be determined. By combining the abnormal coordinate positions of each abnormal chip image with the abnormal type, the abnormal detection information corresponding to the optical image can be obtained.

[0066] In a specific embodiment, after step S170, the following steps are further included: generating a corresponding abnormal icon according to the abnormal type of the abnormal chip image; generating an abnormal detection dot map corresponding to the optical image according to the abnormal icon and the abnormal coordinate positions of each abnormal chip image.

[0067] Further, to improve the convenience for inspectors to obtain inspection information, corresponding abnormal icons can be generated according to the abnormal types of abnormal chip images, and different abnormal types correspond to abnormal icons of different colors and / or different shapes. For example, the abnormal icon corresponding to the abnormal chip image of component abnormality is red, the abnormal icon corresponding to incomplete components is yellow, and the abnormal icon corresponding to wiring abnormality is purple.

[0068] According to the abnormal icon and the abnormal coordinate position of the abnormal chip image, an abnormal detection bitmap corresponding to the optical image can be generated. The abnormal detection bitmap contains the icons of each sensor chip, and the arrangement position of the sensor icons in the abnormal detection point information corresponds to the arrangement position of the sensor chips on the circuit board. The icon of a sensor chip without abnormality is white, and the abnormal icon corresponding to the sensor chip with abnormality is displayed at the corresponding position in the abnormal detection bitmap.

[0069] In the intelligent detection method based on image recognition disclosed in the above embodiments, the method includes: separating the optical image collected by the imaging device to obtain the chip image of each chip, extracting image feature information from each chip image and performing abnormal recognition, integrity recognition and wiring recognition, determining the abnormal type of the abnormal chip image according to the abnormal recognition result, integrity recognition result and wiring recognition result, and combining the abnormal coordinate position and abnormal type of the abnormal chip image to obtain abnormal detection information. The above method can separate the optical image to obtain the chip image of each chip, and separately recognize each chip image to determine whether the chip image is an abnormal chip image, and comprehensively obtain abnormal detection information by combining the abnormal coordinate position and abnormal type of the abnormal chip image, greatly improving the efficiency and reliability of detecting the assembly quality of sensor chips.

[0070] An embodiment of the present invention further provides an intelligent detection device based on image recognition. The intelligent detection device based on image recognition can be configured in a detection device, and the intelligent detection device based on image recognition is used to execute any one of the above embodiments of the intelligent detection method based on image recognition. Specifically, please refer to Figure 3 , Figure 3 which is a schematic block diagram of the intelligent detection device based on image recognition provided by an embodiment of the present invention.

[0071] As Figure 3 shown, the intelligent detection device 100 based on image recognition includes an image segmentation unit 110, an image feature information acquisition unit 120, a first recognition unit 130, a second recognition unit 140, a third recognition unit 150, a determination unit 160, and a combination unit 170.

[0072] An image segmentation unit 110, configured to, if receiving the optical image collected by the imaging device, segment the optical image according to a preset image segmentation rule to obtain a chip image corresponding to each chip in the optical image;

[0073] An image feature information acquisition unit 120, configured to extract corresponding image feature information from each chip image according to a preset image feature extraction model;

[0074] A first recognition unit 130, configured to perform anomaly recognition on each of the image feature information respectively according to a preset component anomaly recognition rule to obtain a component anomaly recognition result indicating whether there is an anomaly;

[0075] A second recognition unit 140, configured to, if the component anomaly recognition result of the image feature information is not an anomaly, perform integrity recognition on the image feature information according to a preset integrity recognition rule to obtain an integrity recognition result indicating whether it is complete;

[0076] A third recognition unit 150, configured to, if the integrity recognition result of the image feature information is complete, perform wiring recognition on the image feature information according to a preset wiring recognition rule to obtain a wiring recognition result indicating whether the wiring is abnormal;

[0077] A determination unit 160, configured to determine an abnormal chip image and a corresponding abnormal type according to the component anomaly recognition result, the integrity recognition result, and the wiring recognition result;

[0078] A combination unit 170, configured to combine the abnormal coordinate positions of each abnormal chip image in the optical image with the corresponding abnormal type to obtain abnormal detection information corresponding to the optical image.

[0079] In the intelligent detection device based on image recognition provided in the embodiments of the present invention, the above-mentioned intelligent detection method based on image recognition is applied to segment the optical image collected by the imaging device to obtain a chip image of each chip, extract image feature information from each chip image and perform anomaly recognition, integrity recognition, and wiring recognition, determine the abnormal type of the abnormal chip image according to the anomaly recognition result, the integrity recognition result, and the wiring recognition result, and combine the abnormal coordinate position of the abnormal chip image with the abnormal type to obtain abnormal detection information. The above method can segment the optical image to obtain a chip image of each chip, and separately identify each chip image to determine whether the chip image is an abnormal chip image, and comprehensively obtain abnormal detection information by combining the abnormal coordinate position of the abnormal chip image with the abnormal type, greatly improving the efficiency and reliability of detecting the assembly quality of sensor chips.

[0080] The above intelligent detection device based on image recognition can be implemented in the form of a computer program, which can run on a computer device as shown in Figure 4 .

[0081] Please refer to Figure 4 . Figure 4 FIG. is a schematic block diagram of a computer device provided by an embodiment of the present invention. The computer device may be a detection device for executing an intelligent detection method based on image recognition to intelligently detect the assembly quality of a sensor chip.

[0082] Referring to Figure 4 , the computer device 500 includes a processor 502, a memory, and a communication interface 505 connected through a communication bus 501. Among them, the memory may include a storage medium 503 and an internal memory 504.

[0083] The storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 can be made to execute an intelligent detection method based on image recognition. Among them, the storage medium 503 can be a volatile storage medium or a non-volatile storage medium.

[0084] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500.

[0085] The internal memory 504 provides an environment for the operation of the computer program 5032 in the storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can be made to execute an intelligent detection method based on image recognition.

[0086] The communication interface 505 is used for network communication, such as providing the transmission of data information, etc. Those skilled in the art can understand that Figure 4 the structure shown in FIG. is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device 500 to which the solution of the present invention is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0087] Among them, the processor 502 is used to run the computer program 5032 stored in the memory to implement the corresponding functions in the above intelligent detection method based on image recognition.

[0088] Those skilled in the art can understand that Figure 4The embodiments of the computer device shown do not constitute a limitation on the specific composition of the computer device. In other embodiments, the computer device may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. For example, in some embodiments, the computer device may include only a memory and a processor. In such an embodiment, the structures and functions of the memory and the processor are the same as those in Figure 4 the shown embodiment and will not be elaborated herein.

[0089] It should be understood that in the embodiments of the present invention, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0090] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps included in the above-mentioned intelligent detection method based on image recognition are implemented.

[0091] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices, apparatuses, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein. Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0092] In several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, or units with the same function can be aggregated into one unit. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Additionally, the displayed or discussed couplings or direct couplings or communication connections to each other can be indirect couplings or communication connections through some interfaces, devices, or units, and can also be in the form of electrical, mechanical, or other connections.

[0093] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the objectives of the embodiments of the present invention.

[0094] In addition, in each embodiment of the present invention, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0095] If the above-mentioned integrated units are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned computer-readable storage medium includes various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), magnetic disks, or optical discs that can store program codes.

[0096] As described above, the above are only the specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. An intelligent detection method based on image recognition, characterized in that: The method is applied to a detection device, the detection device is connected to a camera device through a network to achieve data information transmission, and the method includes: If an optical image acquired by the camera device is received, the optical image is segmented according to a preset image segmentation rule to obtain a chip image corresponding to each chip in the optical image; Extract corresponding image feature information from each chip image according to a preset image feature extraction model; According to the preset component abnormality identification rules, each of the image feature information is respectively identified as abnormal, and a component abnormality identification result is obtained to determine whether the component is abnormal; If the component abnormality identification result of the image feature information is not abnormal, performing integrity identification on the image feature information according to a preset integrity identification rule to obtain an integrity identification result of whether it is complete; If the integrity recognition result of the image feature information is complete, performing wiring recognition on the image feature information according to a preset wiring recognition rule to obtain a wiring recognition result of whether the wiring is abnormal; Determine an abnormal chip image and a corresponding abnormal type according to the component abnormality identification result, the integrity identification result and the wiring identification result; The abnormal coordinate position of each abnormal chip image in the optical image is combined with the corresponding abnormal type to obtain abnormal detection information corresponding to the optical image.

2. The intelligent detection method based on image recognition according to claim 1, characterized in that: The step of segmenting the optical image according to a preset image segmentation rule to obtain a chip image corresponding to each chip in the optical image includes: Determine the corresponding template boundary position and template boundary size according to the positioning mark on the optical image; Adjusting the default cutting template in the image segmentation rule according to the template boundary size to obtain a target cutting template that matches the template boundary size; Covering the target cutting template on the upper layer of the optical image according to the template boundary position; The optical image is cut according to the target cutting template to obtain a chip image.

3. The intelligent detection method based on image recognition according to claim 1 or 2, characterized in that: After the optical image is cut according to the target cutting template to obtain a chip image, the method further includes: Acquire a corresponding two-dimensional code image from the two-dimensional code image cutting area set in the target cutting template; The two-dimensional code image is recognized according to a preset two-dimensional code recognition model to obtain two-dimensional code information corresponding to the optical image.

4. The intelligent detection method based on image recognition according to claim 1, characterized in that: The step of extracting corresponding image feature information from each chip image according to a preset image feature extraction model includes: Extracting regional pixel features from each of the chip images according to the feature extraction region in the image feature extraction model to obtain regional pixel feature information corresponding to each of the chip images; Extracting feature pixel contours from each chip image according to the feature pixel interval in the image feature extraction model to obtain feature pixel contour information corresponding to each chip image; The regional pixel feature information and the feature pixel contour information are combined into corresponding image feature information.

5. The intelligent detection method based on image recognition according to claim 1, characterized in that: The abnormality identification of each of the image feature information is performed according to the preset component abnormality identification rule to obtain the component abnormality identification result of whether it is abnormal, including: Calculate the regional pixel mean corresponding to each component area in the regional pixel feature information of the image feature information; According to the pixel mean value range corresponding to each component area in the component abnormality identification rule, it is determined whether the regional pixel means of each component area are all within the corresponding pixel mean value range to obtain first abnormality judgment information; If the first abnormality judgment result of the image feature information is yes, according to the difference value range corresponding to each component area in the component abnormality identification rule, it is judged whether the pixel difference between the pixel value of the pixel point in each component area and the regional pixel mean is within the corresponding difference value range, and a second abnormality judgment result is obtained; If the second abnormality judgment result of the image feature information is yes, obtaining a normal abnormality recognition result; If the first abnormality judgment result of the image feature information is no or the second abnormality judgment result is no, an abnormal abnormality recognition result is obtained.

6. The intelligent detection method based on image recognition according to claim 1, characterized in that: The step of performing integrity identification on the image feature information according to a preset integrity identification rule to obtain an integrity identification result of whether the image feature information is complete includes: Determine whether the maximum curvature of the contour line of each component contour area in the feature pixel contour information of the image feature information is located within the curvature interval corresponding to each component contour area in the integrity identification rule, and obtain a first integrity judgment result; If the first integrity judgment result is yes, calculating the angle value between the contour line of each component contour area in the characteristic pixel contour information and the image frame of the corresponding chip image; Determine whether the angle values ​​of the characteristic pixel contour information are all not greater than the angle threshold in the integrity identification rule, and obtain a second integrity determination result; If the second integrity judgment result of the image feature information is yes, obtaining a complete integrity recognition result; If the first integrity judgment result of the image feature information is no or the second integrity judgment result is no, an incomplete integrity recognition result is obtained.

7. The intelligent detection method based on image recognition according to claim 1, characterized in that: After combining the abnormal coordinate position of each abnormal chip image at the optical image with the corresponding abnormal type to obtain the abnormality detection information corresponding to the optical image, the method further includes: Generate a corresponding abnormal icon according to the abnormal type of the abnormal chip image; An abnormality detection point map corresponding to the optical image is generated according to the abnormal icon and the abnormal coordinate position of each abnormal chip image.

8. An intelligent detection device based on image recognition, characterized in that: The intelligent detection device based on image recognition is used to execute the intelligent detection method based on image recognition according to any one of claims 1 to 7. The device is configured in a detection device, and the detection device is connected to a camera device through a network to realize the transmission of data information. The device includes: An image segmentation unit, configured to segment the optical image according to a preset image segmentation rule upon receiving the optical image acquired by the camera device, so as to obtain a chip image corresponding to each chip in the optical image; An image feature information acquisition unit, used to extract corresponding image feature information from each chip image according to a preset image feature extraction model; A first recognition unit is used to perform abnormality recognition on each of the image feature information according to a preset component abnormality recognition rule to obtain a component abnormality recognition result indicating whether the component is abnormal; A second identification unit is used to perform integrity identification on the image feature information according to a preset integrity identification rule to obtain an integrity identification result of whether the image feature information is complete if the component abnormality identification result of the image feature information is not abnormal; A third identification unit is used to, if the integrity identification result of the image feature information is complete, perform wiring identification on the image feature information according to a preset wiring identification rule to obtain a wiring identification result of whether the wiring is abnormal; A determination unit, configured to determine an abnormal chip image and a corresponding abnormal type according to the component abnormality identification result, the integrity identification result and the wiring identification result; The combining unit is used to combine the abnormal coordinate position of each abnormal chip image in the optical image with the corresponding abnormal type to obtain the abnormal detection information corresponding to the optical image.

9. A computer device, characterized in that: The device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement the steps of the intelligent detection method based on image recognition described in any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the intelligent detection method based on image recognition as described in any one of claims 1 to 7 are implemented.

Images