Mobile phone hole area defect detection method, system, equipment and medium
By automatically identifying the hole area defects in the mobile phone, the problem of large errors in artificial visual detection is solved, and high accuracy and high resolution hole area defect detection is achieved, ensuring the reliability of the detection results.
Patent Information
- Application Number
- CN202411028523.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-22
AI Technical Summary
In the prior art, artificial visual detection of defects in the mobile phone hole area is susceptible to environmental influences, resulting in large errors and low accuracy of detection results.
By obtaining the defect detection requirements of mobile phone hole area, determining the detection environment, and dividing the hole area into multiple areas to be photographed, using the camera to shoot and identify, automatically identifying the defect type of hole area, eliminating human eye recognition errors, and improving the accuracy of detection results sex.
Automatic identification of defects in the hole area of the mobile phone is realized, the accuracy and resolution of the detection results are improved, the integrity and accuracy of defect information in the hole area is ensured, and manual errors are reduced.
Smart Images

Figure CN119006383A_ABST
Abstract
Description
Background Art
[0002] In the development history of full-screen smartphones, there are three types that can be called classics: bangs screen, water drop screen, and hole screen. The characteristic of their continuous development is that they cover the screen less and less.
[0003] At present, the full-screen forms on the market have their own advantages and disadvantages: ① The pop-up screen is the most extreme full-screen, but the structure is complex and the thickness is increased, and the space of mobile phones in the 5G era is very tight; ② The water drop screen has a small irregular area, but it is aesthetically fatigued; ③ The camera under the screen is not mature at present; ④ The hole screen has a high screen-to-body ratio. 5G mobile phones can remain thin and light without compromising the front camera. The technology is mature, but the visual area is irregular. There are still regrets, but it is already the first choice. It can be said that the hole screen is a derivative and evolution of the fusion of users' demand for high screen-to-body ratio and user experience, and it is also a transition to the pursuit of full-screen under-screen camera. With the market's pursuit of full-screen, the full-screen with a hole has occupied the mainstream of the market. In order to take into account some functions of the mobile phone, a screen with a small hole reserved on the inner screen of the mobile phone screen can be defined as a hole screen. Based on the design concept of the mobile phone and the limitations of the manufacturing process, this hole can be located at the left and right corners of the screen or in the middle of the screen. The advantage of the punch hole screen is that the screen ratio on the front is indeed larger than that of the bangs and water drop screen, and the abruptness is not so serious. When playing games or watching videos, the displayed content on the screen is further increased, and the content is less blocked. However, its disadvantage is also obvious, that is, the quality control is prone to problems, resulting in blackening at the bottom or edge of the screen, which is particularly obvious on a white background.
[0004] At present, it is particularly important to control the quality of punch hole screens. Common defects in the hole area of punch hole screens are cracks and rainbow patterns, which seriously affect the quality of punch hole screens. The most common inspection method used by general screen manufacturers is manual visual inspection. However, the human eye is easily affected by the environment, which makes the error generated in the manual visual inspection process larger, resulting in low accuracy of the inspection results. Summary of the invention
[0005] In order to overcome the problem that the human eye is easily affected by the environment, resulting in large errors in the manual visual inspection process and low accuracy of the inspection results, the present application provides a mobile phone hole area defect detection method, system, equipment and medium.
[0006] In the first aspect, in order to solve the above technical problems, the present application provides a method for detecting defects in a mobile phone hole area, comprising:
[0007] Obtain the hole area defect detection requirements of the mobile phone to be tested;
[0008] Based on the hole area defect detection requirements, determine the testing environment for the mobile phone to be tested;
[0009] Divide the area where the hole area of the mobile phone to be tested is located into multiple areas to be photographed;
[0010] Based on the detection environment, the shooting area is photographed to obtain a photographed image;
[0011] The defect types of the hole area are obtained by identifying multiple captured images.
[0012] In a second aspect, the present application also provides a mobile phone hole area defect detection system, comprising:
[0013] An acquisition module is used to obtain the hole area defect detection requirements of the mobile phone to be tested;
[0014] A determination module, used to determine a detection environment for detecting the mobile phone to be tested based on the hole area defect detection requirements;
[0015] A division module, used to divide the area where the hole area of the mobile phone to be tested is located into multiple areas to be photographed;
[0016] A shooting module, used to shoot the shooting area based on the detection environment to obtain a shot image;
[0017] The recognition module is used to recognize multiple captured images and obtain the defect type of the hole area.
[0018] In a third aspect, the present application also provides a computing device, including a memory, a processor, and a program stored in the memory and running on the processor, and when the processor executes the program, the steps of a method for detecting defects in a mobile phone hole area as described above are implemented.
[0019] In a fourth aspect, the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes the steps of a method for detecting defects in a mobile phone hole area.
[0020] The beneficial effects of the present application are as follows: by determining the detection environment based on the hole area defect detection requirements of the mobile phone to be tested, and in this detection environment, using a camera to respectively shoot multiple areas to be shot divided from the hole area of the mobile phone to be tested, obtaining multiple shot images, identifying the defect type of the hole area in the multiple shot images, and realizing automatic identification of the defect type of the hole area of the mobile phone to be tested, eliminating the error caused by human eye recognition, and improving the accuracy of the detection results. At the same time, compared with the prior art of shooting the entire area where the hole area is located to obtain an overall image, the present application obtains a shot image by shooting the hole area of the mobile phone to be tested in different areas, which not only has a higher resolution than the resolution of the overall image, improving the accuracy of the effective information in the shot image, but also ensures that the defect information in the area where the hole area is located can be fully captured, so that the quantity and accuracy of the effective defect information in the multiple shot images for subsequent detection can meet the recognition requirements, thereby further improving the accuracy of the detection results. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 A schematic diagram of a process for detecting defects in a hole area of a mobile phone according to the present application;
[0022] Figure 2 A schematic diagram of the structure of a camera configured based on height requirements in this application;
[0023] Figure 3 This is a device structure diagram corresponding to a method for detecting defects in a hole area of a mobile phone in this application;
[0024] Figure 4 A schematic diagram of an acquisition trajectory for image acquisition in this application;
[0025] Figure 5 This is a structural schematic diagram of a mobile phone hole area defect detection system for the present application. DETAILED DESCRIPTION
[0026] The following examples are provided to further explain and supplement the present application and do not constitute any limitation to the present application.
[0027] The following describes a method, system, device and medium for detecting defects in a mobile phone hole area according to an embodiment of the present application in conjunction with the accompanying drawings.
[0028] A method for detecting defects in a mobile phone hole area is disclosed in an embodiment of the present application. The method is applied to a terminal device. In the present application scheme, the terminal device is used as the execution subject to illustrate the present application scheme. The terminal device is used to execute the steps of a method for detecting defects in a mobile phone hole area.
[0029] like Figure 1 As shown, the present application provides a method for detecting defects in a mobile phone hole area, comprising:
[0030] Step S11, obtaining the hole area defect detection requirements of the mobile phone to be tested;
[0031] Step S12, based on the hole area defect detection requirements, determining the detection environment for the mobile phone to be tested;
[0032] Step S13, dividing the area where the hole area of the mobile phone to be tested is located into a plurality of areas to be photographed;
[0033] Step S14, photographing the area to be photographed based on the detection environment to obtain a photographed image;
[0034] Step S15, identifying the multiple captured images to obtain the defect type of the hole area.
[0035] A method for detecting defects in the hole area of a mobile phone of the present embodiment determines the detection environment based on the hole area defect detection requirements of the mobile phone to be tested, and in this detection environment, uses a camera to respectively shoot multiple areas to be shot divided from the hole area of the mobile phone to be tested, obtains multiple shot images, identifies the defect types of the hole area in the multiple shot images, and realizes automatic identification of the defect types of the hole area of the mobile phone to be tested, eliminates the errors caused by human eye recognition, and improves the accuracy of the detection results. At the same time, compared with the prior art of shooting the entire area where the hole area is located to obtain an overall image, the present application obtains a shot image by shooting the hole area of the mobile phone to be tested in different areas, which not only has a higher resolution than the resolution of the overall image, improves the accuracy of the effective information in the shot image, but also ensures that the defect information in the area where the hole area is located can be fully captured, so that the quantity and accuracy of the effective defect information in the multiple shot images to be tested subsequently can meet the recognition requirements, thereby further improving the accuracy of the detection results. Among them, the defect types include crack types and rainbow pattern types.
[0036] Optionally, the hole area defect detection requirement includes a hole area crack detection requirement and a hole area rainbow pattern detection requirement; based on the hole area defect detection requirement, determining a detection environment for detecting the mobile phone to be tested, including:
[0037] Based on the hole area crack detection requirement, a first detection environment corresponding to a ring light source for detecting the mobile phone to be tested is obtained;
[0038] Based on the hole area rainbow pattern detection requirement and the first detection environment, a second detection environment corresponding to the point light source for detecting the mobile phone to be tested is obtained;
[0039] The first testing environment and the second testing environment are used as testing environments for testing the mobile phone to be tested.
[0040] In this embodiment, since the contrast of the crack in the hole area is high when it is in the dark field ambient light, it is easy to be photographed and imaged, and the dark field ambient light is often obtained by setting a ring light source. Based on the detection requirements of the crack in the hole area, it can be understood that the ambient light condition required by the hole area of the mobile phone to be tested is the dark field ambient light, so that the first detection environment corresponding to the ring light source for detecting the mobile phone to be tested is directly obtained, which is convenient for obtaining the dark field ambient light later. At the same time, since the rainbow pattern in the hole area has a high contrast when it is in the bright field ambient light, it is easy to be photographed and imaged, and the bright field ambient light is often obtained by setting a point light source. Based on the detection requirements of the rainbow pattern in the hole area, it can be understood that the ambient light condition required by the hole area of the mobile phone to be tested is the bright field ambient light, so that the second detection environment corresponding to the point light source for detecting the mobile phone to be tested is directly obtained, which is convenient for obtaining the bright field ambient light later. The first detection environment and the second detection environment are used as the detection environment for detecting the mobile phone to be tested, which is convenient for the subsequent shooting and identification of high-contrast cracks and high-contrast rainbow patterns in the hole area in this detection environment. The ring light source is set at 30° downward horizontally, the installation height of the ring light source is 75mm, the inner diameter of the ring light source is 35mm, and the outer diameter is 70mm. Under this setting, the ambient light emitted by the ring light source can make the contrast of the cracks in the hole area of the mobile phone under test in the ambient light reach the highest. The camera position requirements include height requirements, such as Figure 2 As shown, the height of the camera after being set according to the height requirement is 75mm.
[0041] In some embodiments, when the hole area of the mobile phone to be tested is in dark field ambient light, only the light scattered by the grooves caused by the cracks in the hole area can be detected, and the background light of the smooth surface of the hole area is blocked or attenuated, thereby improving the crack contrast of the hole area, improving the effect of photographing and imaging the cracks, and can improve the accuracy of the crack type obtained by subsequent image recognition. When the hole area of the mobile phone to be tested is in bright field ambient light, the rainbow pattern in the hole area is under a bright background of uniform illumination, and the rainbow pattern on the surface of the hole area and the background absorb, transmit and reflect the bright light, enhancing the color difference between the rainbow pattern and the background, thereby improving the contrast of the rainbow pattern in the hole area, improving the photographing and imaging effect of the rainbow pattern, and can improve the accuracy of the rainbow pattern type obtained by subsequent image recognition.
[0042] Optionally, based on the hole area crack detection requirement, obtaining a first detection environment corresponding to a ring light source for detecting the mobile phone to be tested includes:
[0043] Based on the hole area crack detection requirements, determine the first setting requirements of the ring light source for detecting the mobile phone to be tested; wherein the first setting requirements include the dark field parameter setting requirements and position setting requirements of the ring light source;
[0044] Based on the first setting requirement, a position requirement of a camera for shooting the mobile phone to be tested is obtained;
[0045] The environment constructed based on the first setting requirement and the position requirement is used as the first detection environment corresponding to the ring light source.
[0046] In this embodiment, based on the crack detection requirements of the hole area, the dark field ambient light required for crack detection can be understood, and the requirement for setting the ring light source when obtaining the dark field ambient light is used as the first setting requirement. Since the camera that shoots the mobile phone to be tested needs to shoot the red area of the mobile phone to be tested that is in the irradiation area of the dark field ambient light emitted by the ring light source, the positions of the camera and the ring light source are relatively fixed, so when the position of the ring light source is known, the position of the camera can be directly known to directly obtain the position requirement of the camera. After the ring light source is set according to the first setting requirement and the camera is set according to the position requirement, the first detection environment is obtained. In the first detection environment, when the area to be shot of the hole area of the mobile phone to be tested is in the target shooting area of the camera, the area to be shot of the hole area of the mobile phone to be tested is irradiated and photographed, and a photographed image containing the cracks in the mobile phone to be tested and the contrast that meets the recognition requirements can be obtained, thereby improving the accuracy of subsequent recognition and the crack type obtained by recognition.
[0047] Optionally, based on the hole area rainbow pattern detection requirement and the first detection environment, obtaining a second detection environment corresponding to the point light source for detecting the mobile phone to be tested includes:
[0048] Based on the hole area rainbow pattern detection requirement and the first setting requirement, determine the second setting requirement of the point light source for detecting the mobile phone to be tested;
[0049] The environment constructed based on the second setting requirement and the position requirement is used as the second detection environment corresponding to the point light source.
[0050] Optionally, based on the hole area rainbow pattern detection requirement and the first setting requirement, determining a second setting requirement of a point light source for detecting the mobile phone to be tested includes:
[0051] Based on the first setting requirement, determine the relative position requirements of the point light source and the ring light source for testing the mobile phone to be tested;
[0052] Based on the rainbow pattern detection requirements in the hole area, determine the bright field parameter setting requirements of the point light source;
[0053] The relative position requirement and the parameter setting requirement are used as the second setting requirement for the point light source for detecting the mobile phone to be tested.
[0054] In this embodiment, based on the requirements for rainbow pattern detection in the hole area, the bright field ambient light required for rainbow pattern detection can be understood. At the same time, since the position of the camera is determined by the position of the ring light source, that is, the target shooting area of the camera is fixed, in order for the bright field ambient light emitted by the power light to illuminate the shooting area during shooting, the positions of the point light source, the ring light source, and the camera are relatively fixed. Therefore, the requirements for setting the ring light source when obtaining the dark field ambient light and the relative position requirements of the point light source relative to the ring light source are used as the second setting requirements. After the point light source is set according to the second setting requirements and the camera is set according to the position requirements, a second detection environment is obtained. In the second detection environment, when the area to be shot of the hole area of the mobile phone to be tested is in the target shooting area of the camera, the area to be shot of the hole area of the mobile phone to be tested is illuminated and photographed, and a photographed image containing the rainbow patterns in the mobile phone to be tested whose contrast meets the recognition requirements can be obtained, thereby improving the accuracy of subsequent recognition and the type of rainbow patterns obtained by recognition.
[0055] In some embodiments, since the size of the cracks and rainbow patterns in the hole area of the screen of the mobile phone to be tested is very small, the shooting accuracy must also match it. Therefore, the body of the camera can be an area array CCD camera, and the area array CCD camera can be a 25M color camera of Hikvision. The CCD of the 25M color camera is square (5120×5120), the camera pixel size is 2.5μm, and the target surface size is 1 inch, which improves the utilization rate and shooting accuracy of the camera's shooting field of view. The lens of the camera can be a telecentric lens, the telecentric lens can be 3X times, and the target surface size of the telecentric lens is 1 inch, so that it is adapted to the 25M color camera. The telecentric lens can make each chip in the middle part and the edge have clear images. The main feature is that it can overcome perspective errors and has the characteristics of high resolution, high contrast, large depth of field, and low distortion. The use of a telecentric lens can well overcome the drum distortion and pincushion distortion caused by an ordinary lens, and obtain high-quality images at the edge of the image. The pixel accuracy of the telecentric lens is 0.83μm.
[0056] When detecting rainbow defects, the use of color cameras has good imaging contrast and can better locate and detect defects through color information. Since the cracks in the hole area are submicron cracks with a physical size of less than 1um, a high-precision visual system (25M color camera + 3X telecentric lens) is required to meet the detection requirements when detecting crack defects.
[0057] In some embodiments, Figure 3As shown, a corresponding device for implementing a method for detecting defects in the hole area of a mobile phone includes a camera, a point light source, a ring light source, a distance measurement module and a mobile storage table, and the device also includes a mobile phone to be tested, a motor and a slide; the slide is connected to the mobile storage table, the slide is slidably connected to the camera through a motor, and the distance measurement module, the point light source and the ring light source are all arranged on the camera; wherein the center position of the ring light source is in the same horizontal plane as the mirror position of the camera lens. The mobile phone to be tested is placed on the mobile slide. When in use, the ring light source is set according to the first setting requirement, the point light source is set according to the second setting requirement, and the position of the motor is adjusted to meet the position requirement by the motor, and the mobile storage table is controlled to drive the mobile phone to be tested to move in the horizontal x-axis and y-axis directions, so that each area to be photographed in the hole area of the mobile phone to be tested enters the target shooting area of the camera in turn, and the time interval is set according to 80% of the maximum frame rate of the camera as the shooting time interval for collecting images of the target shooting area. Each time the area to be photographed is in the target photographing area, the point light source is first turned off and the ring light source is turned on to photograph the first photographing image corresponding to the crack defect, and then the ring light source is turned off and the point light source is turned on to photograph the second photographing image corresponding to the rainbow defect.
[0058] The point light source includes a coaxial point light source and a reserved point light source. The coaxial point light source vertically illuminates the target shooting area of the camera. The reserved point light source is set at an angle, and there are 4 of them. After the 4 reserved point light sources are lit, the illuminated area is the target shooting area of the camera. When the coaxial point light source is damaged or the brightness does not meet the requirements for enhancing the contrast of the rainbow pattern, the 4 reserved point light sources are used to light up at the same time to provide bright field lighting that can meet the requirements for enhancing the contrast of the rainbow pattern. The ranging module can test the distance from the camera to the shooting area of the mobile phone to be tested according to the tolerance of different mobile phone models and the thickness of the product cover. The position of the camera can be adjusted in real time according to the distance to meet the position requirements of the camera when shooting, thereby adjusting the image clarity of the shooting area of the mobile phone to be tested, and ensuring that the captured image can be clearly imaged. The power supply of the camera uses a camera power supply controller to realize a DC 24V voltage supply; when triggering the camera to shoot, a 12V DC voltage is used. After receiving the signal that the shooting area of the hole area of the mobile phone to be tested moves to the target shooting area of the camera, the camera starts image acquisition.
[0059] Optionally, photographing the area to be photographed based on the detection environment to obtain a photographed image includes:
[0060] The camera is set based on the position requirements to obtain the target shooting area;
[0061] When the point light source is in an off state, the ring light source is set based on a first setting requirement to obtain a first ambient light;
[0062] When the area to be photographed is in the first ambient light and is within the target photographing area, photographing the area to be photographed to obtain a first photographed image corresponding to the hole area crack detection requirement;
[0063] When the ring light source is in an off state, the point light source is set based on a second setting requirement to obtain a second ambient light;
[0064] When the area to be photographed is in the second ambient light and is located in the target photographing area, photographing the area to be photographed to obtain a second photographed image corresponding to the hole area rainbow pattern detection requirement;
[0065] The first captured image and the second captured image are taken as captured images.
[0066] In this embodiment, when the point light source is in the off state, the ring light source is set based on the first setting requirement to obtain the first ambient light, which can improve the crack contrast of the hole area of the mobile phone to be tested in the first ambient light, so that the effective information of the crack in the first captured image can meet the subsequent recognition requirements. When the ring light source is in the off state, the point light source is set based on the second setting requirement to obtain the second ambient light, which can improve the rainbow pattern contrast of the hole area of the mobile phone to be tested in the second ambient light, so that the effective information of the rainbow pattern in the second captured image can meet the subsequent recognition requirements. The first captured image and the second captured image are used as captured images, and multiple captured images are recognized, that is, multiple first captured images and multiple second captured images are recognized respectively, which can improve the accuracy of the recognized crack types and rainbow pattern types.
[0067] In some embodiments, the camera needs to collect the bright field image corresponding to the point light source and the dark field image corresponding to the ring light source at the same point (it can be to collect the dark field image first, and then collect the bright field image). In order to improve the detection efficiency, the area to be photographed of the mobile phone to be tested needs to quickly collect the bright field image after the dark field image acquisition is completed, so that the acquisition time of the two images can be minimized, reducing the total shooting time for shooting all the areas to be photographed in the hole area. In order to achieve this goal, this application uses a new type of light source controller to control. The control process of the light source controller is that after the area to be photographed in the hole area moves to the target shooting area of the camera, the terminal device PLC (Programmable Logic Controller) sends a signal to the light source controller, and the light source controller controls the ring light source (dark field light source) and the point light source (bright field light source) to light up respectively, and at the same time, when each light source is lit, it gives the camera a trigger signal for image acquisition. The interval duration of the trigger signal of the bright and dark fields can be calculated using the maximum frame rate of the camera. In order to make the captured image more stable and avoid the risk of losing the image, 80% of the maximum frame rate can be used to set the time interval to achieve stable acquisition of the captured image.
[0068] Optionally, the area where the hole area of the mobile phone to be tested is located is divided into a plurality of areas to be photographed, including:
[0069] Obtain the shooting field of view of the camera used to shoot the mobile phone to be tested, and obtain the shooting area corresponding to the camera;
[0070] The area where the hole area of the mobile phone to be tested is located is divided into a plurality of areas to be photographed, so that the area of the area to be photographed is less than or equal to the shooting area.
[0071] In this embodiment, the areas of the multiple areas to be photographed obtained by dividing the area where the hole area of the mobile phone to be tested is smaller than the shooting area corresponding to the camera's shooting field of view, and each area to be photographed is subsequently photographed to obtain a corresponding captured image. Compared with the prior art in which the entire area where the hole area is located is photographed as a whole to obtain an overall image, the captured image of the present application not only has a higher resolution than the resolution of the overall image, improving the accuracy of the effective information in the captured image, but also ensures that the defect information in the area where the hole area is located can be fully captured, so that the amount and accuracy of the effective defect information in the multiple captured images for subsequent testing can meet the recognition requirements, thereby further improving the accuracy of the test results.
[0072] When photographing the hole area of the mobile phone to be tested, this embodiment divides the area where the hole area of the mobile phone to be tested is located into multiple areas to be photographed, each of which is smaller than the shooting area of the camera's shooting field of view. When photographing multiple areas to be photographed, the camera adopts a fixed shooting method, that is, according to the multiple areas to be photographed in the hole area, the trajectory of image acquisition is formulated. Taking a circular hole with a diameter of 5 mm as an example (the effective detection range is a circular area with a diameter of 8 mm and the center of the circular hole as the origin), there are a total of 4 areas to be photographed, such as Figure 4 As shown, the area where the hole area of the mobile phone to be tested is located is divided into 4 areas to be photographed ①②③④ with the center of the arc as the center point, and the trajectory of image acquisition is shown in the direction of the arrow in the figure. When one of the areas to be photographed in ①②③④ is in the target shooting area where the camera's shooting field of view is located, the camera is used to perform bright field shooting corresponding to the point light source and dark field shooting corresponding to the ring light source, respectively, to obtain the corresponding first shooting image and second shooting image, thereby obtaining the shooting image. Among them, the camera body can be a Hikvision 25M color camera, and the camera's shooting field of view size = CCD size × system accuracy = 5120 × 0.83 = 4.3mm, that is, the size of the shot image FOV = 4.3 × 4.3mm.
[0073] Optionally, multiple captured images are identified to obtain defect types of the hole area, including:
[0074] Inputting a plurality of captured images into a preset deep learning network, and combining the plurality of captured images into a target image; wherein the target image includes a hole area, and the target image includes a crack image of the hole area and a rainbow pattern image of the hole area;
[0075] Extract image features of the hole area in the target image, and obtain defect information of the hole area based on the image information;
[0076] The defect type of the hole area is obtained based on the defect information.
[0077] In this embodiment, the preset deep learning network is a model that meets the classification requirements for the defect recognition accuracy and defect type classification accuracy of the hole area after pre-training. Then, multiple captured images are combined into a hole area crack image and a hole area rainbow pattern image through the preset deep learning network, and feature recognition is performed on the hole area crack image and the hole area rainbow pattern image to determine the crack features existing in the hole area in the hole area crack image, and the rainbow pattern features existing in the hole area in the hole area rainbow pattern image. Through the crack features and rainbow pattern features, the respectively matching crack types and rainbow pattern types can be found in the preset defect matching table. In this way, in the preset deep learning network, through feature extraction and preset table matching, compared with manual judgment of the defect type of the hole area of the tested mobile phone, not only can the recognition efficiency be accelerated, but also the errors caused by human eye recognition can be avoided, thereby improving the accuracy of the crack defect type.
[0078] In some embodiments, the preset deep learning network is an algorithm model obtained by training using a labeled training data set. During the training process, the network parameters or network structure in the algorithm model are adjusted to improve the generalization ability and accuracy of the algorithm model, so that the recognition accuracy of the preset deep learning network for the captured images can meet the requirements.
[0079] The present application is an optical method for automated detection of cracks and rainbow patterns in the hole area of the screen based on machine vision, to replace the common manual detection in hole area detection. It mainly involves the content of the optical solution of automated equipment in machine vision. Among them, the system of the optical solution consists of three parts: camera, lens, and light source. The selection of light source and camera, lens, and the lighting scheme of the light source directly determine the imaging quality of the captured image, which in turn affects the stability and other performance of the system. The present application uses multi-spectral imaging technology to collect images of two light fields, bright field and dark field, respectively. The dark field is used to detect cracks, and the bright field is used to detect rainbow patterns, to achieve accurate imaging of cracks and rainbow patterns, and to achieve stable shooting and detection of cracks and rainbow pattern defects in the hole area, thereby ensuring the delivery quality of the screen hole area. The optical solution used in the present application can be used for surface detection of various types of hole area cracks and rainbow patterns, including circular, runway hole type, circular + runway hole type, and other products.
[0080] Compared with the manual visual method, the method of the present application is a mobile phone hole area defect detection technology based on machine vision. As a high-precision, flexible, real-time, non-contact detection technology, it can be widely studied and applied, and also provides a new development direction for defect detection. Automatic detection technology mainly revolves around machine vision, and has been widely developed and applied with its advantages of high efficiency, high precision, high reliability, non-contact and strong observability. The detection method of the present application has the following beneficial effects: (1) high detection reliability, which can realize automatic detection of defects according to different sizes and hole types of mobile phone hole areas; (2) high detection efficiency, which can complete the detection of a product within 3 seconds, and the detection speed is several times that of manual detection. Machine vision detection can achieve long-term stable operation, which greatly promotes the production efficiency of enterprises; (3) high accuracy, the results of manual visual detection are subjective, and individual differences and human eye fatigue will affect the detection results. Machine vision detection is objective, and can output specific detection values and results, which is convenient and flexible to modify the detection standards; (4) reduce the labor intensity of workers; (5) help enterprises save a certain amount of human operating costs.
[0081] like Figure 5 As shown, the present application provides a mobile phone hole area defect detection system, comprising:
[0082] An acquisition module is used to obtain the hole area defect detection requirements of the mobile phone to be tested;
[0083] A determination module, used to determine a detection environment for detecting the mobile phone to be tested based on the hole area defect detection requirements;
[0084] A division module, used to divide the area where the hole area of the mobile phone to be tested is located into multiple areas to be photographed;
[0085] A shooting module, used to shoot the shooting area based on the detection environment to obtain a shot image;
[0086] The recognition module is used to recognize multiple captured images and obtain the defect type of the hole area.
[0087] Optionally, a module is determined, specifically for:
[0088] Based on the hole area crack detection requirement, a first detection environment corresponding to a ring light source for detecting the mobile phone to be tested is obtained;
[0089] Based on the hole area rainbow pattern detection requirement and the first detection environment, a second detection environment corresponding to the point light source for detecting the mobile phone to be tested is obtained;
[0090] The first testing environment and the second testing environment are used as testing environments for testing the mobile phone to be tested.
[0091] Optionally, a module is determined, specifically for:
[0092] Based on the hole area crack detection requirements, determine the first setting requirements of the ring light source for detecting the mobile phone to be tested;
[0093] Based on the first setting requirement, a position requirement of a camera for shooting the mobile phone to be tested is obtained;
[0094] The environment constructed based on the first setting requirement and the position requirement is used as the first detection environment corresponding to the ring light source.
[0095] Optionally, a module is determined, specifically for:
[0096] Based on the hole area rainbow pattern detection requirement and the first setting requirement, determine the second setting requirement of the point light source for detecting the mobile phone to be tested;
[0097] The environment constructed based on the second setting requirement and the position requirement is used as the second detection environment corresponding to the point light source.
[0098] Optionally, the shooting module is specifically used to:
[0099] The camera is set based on the position requirements to obtain the target shooting area;
[0100] When the point light source is in an off state, the ring light source is set based on a first setting requirement to obtain a first ambient light;
[0101] When the area to be photographed is in the first ambient light and is within the target photographing area, photographing the area to be photographed to obtain a first photographed image corresponding to the hole area crack detection requirement;
[0102] When the ring light source is in an off state, the point light source is set based on a second setting requirement to obtain a second ambient light;
[0103] When the area to be photographed is in the second ambient light and is located in the target photographing area, photographing the area to be photographed to obtain a second photographed image corresponding to the hole area rainbow pattern detection requirement;
[0104] The first captured image and the second captured image are taken as captured images.
[0105] Optionally, the modules are divided to:
[0106] Obtain the shooting field of view of the camera used to shoot the mobile phone to be tested, and obtain the shooting area corresponding to the camera;
[0107] The area where the hole area of the mobile phone to be tested is located is divided into a plurality of areas to be photographed, so that the area of the area to be photographed is less than or equal to the shooting area.
[0108] Optionally, the identification module is specifically used to:
[0109] Inputting a plurality of captured images into a preset deep learning network, and combining the plurality of captured images into a target image; wherein the target image contains a hole area;
[0110] Extract image features of the hole area in the target image, and obtain defect information of the hole area based on the image information;
[0111] The defect type of the hole area is obtained based on the defect information.
[0112] A computing device according to an embodiment of the present application includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, some or all steps of the above-mentioned method for detecting defects in the hole area of a mobile phone are implemented.
[0113] Among them, the computing device can be a computer, and correspondingly, its program is computer software. The above-mentioned parameters and steps in a computing device of the present application can refer to the parameters and steps in an embodiment of a mobile phone hole area defect detection method mentioned above, and will not be repeated here.
[0114] In an embodiment of the present application, a computer-readable storage medium is provided, in which instructions are stored. When the instructions are executed, the steps of the above-mentioned method for detecting defects in a mobile phone hole area are executed.
[0115] The computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.
[0116] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method of the embodiment of the present disclosure. The aforementioned computer-readable storage medium may be a non-transitory computer-readable storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes, or a transient computer-readable storage medium.
[0117] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0118] Those skilled in the art know that the present application can be implemented as a system, method or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, generally referred to as "module" or "system" herein. In addition, in some embodiments, the present application can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code. Computer-readable storage media can be, for example, but not limited to - electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above.
[0119] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0120] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. A method for detecting defects in a mobile phone hole area, characterized in that: include: Obtain the hole area defect detection requirements of the mobile phone to be tested; Based on the hole area defect detection requirement, determining a detection environment for detecting the mobile phone to be tested; Divide the area where the hole area of the mobile phone to be tested is located into a plurality of areas to be photographed; Based on the detection environment, the area to be photographed is photographed to obtain a photographed image; The plurality of captured images are identified to obtain the defect type of the hole area.
2. The method according to claim 1, characterized in that The hole area defect detection requirement includes a hole area crack detection requirement and a hole area rainbow pattern detection requirement; the determining of a detection environment for detecting the mobile phone to be tested based on the hole area defect detection requirement includes: Based on the hole area crack detection requirement, a first detection environment corresponding to a ring light source for detecting the mobile phone to be tested is obtained; Based on the hole area rainbow pattern detection requirement and the first detection environment, a second detection environment corresponding to the point light source for detecting the mobile phone to be tested is obtained; The first detection environment and the second detection environment are used as detection environments for detecting the mobile phone to be tested.
3. The method according to claim 2, characterized in that The obtaining, based on the hole area crack detection requirement, a first detection environment corresponding to a ring light source for detecting the mobile phone to be tested, comprises: Based on the hole area crack detection requirement, determine a first setting requirement of a ring light source for detecting the mobile phone to be tested; Based on the first setting requirement, obtaining a position requirement of a camera for photographing the mobile phone to be tested; An environment constructed based on the first setting requirement and the position requirement is used as a first detection environment corresponding to the ring light source.
4. The method according to claim 3, characterized in that The obtaining, based on the hole area rainbow pattern detection requirement and the first detection environment, a second detection environment corresponding to the point light source for detecting the mobile phone to be tested comprises: Determine a second setting requirement for a point light source for detecting the mobile phone to be tested based on the hole area rainbow pattern detection requirement and the first setting requirement; An environment constructed based on the second setting requirement and the position requirement is used as a second detection environment corresponding to the point light source.
5. The method according to claim 2, characterized in that: The step of photographing the area to be photographed based on the detection environment to obtain a photographed image includes: The camera is set based on the position requirement to obtain a target shooting area; When the point light source is in an off state, the ring light source is set based on the first setting requirement to obtain a first ambient light; When the area to be photographed is in the first ambient light and is within the target photographing area, photographing the area to be photographed to obtain a first photographed image corresponding to the hole area crack detection requirement; When the ring light source is in an off state, the point light source is set based on the second setting requirement to obtain a second ambient light; When the area to be photographed is in the second ambient light and is located in the target photographing area, photographing the area to be photographed to obtain a second photographed image corresponding to the hole area rainbow pattern detection requirement; The first captured image and the second captured image are used as captured images.
6. The method according to any one of claims 1 to 5, characterized in that: The step of dividing the area where the hole area of the mobile phone to be tested is located into a plurality of areas to be photographed includes: Obtaining the shooting field of view of the camera that shoots the mobile phone to be tested, and obtaining the shooting area corresponding to the camera; The area where the hole area of the mobile phone to be tested is located is divided into a plurality of areas to be photographed, so that the area of the areas to be photographed is less than or equal to the photographing area.
7. The method according to any one of claims 1 to 5, characterized in that: The identifying of the plurality of captured images to obtain the defect type of the hole area includes: Inputting the plurality of captured images into a preset deep learning network, and combining the plurality of captured images into a target image; wherein the target image contains the hole area; Extracting image features of the hole area in the target image, and obtaining defect information of the hole area based on the image information; The defect type of the hole area is obtained based on the defect information.
8. A mobile phone hole area defect detection system, characterized in that: include: An acquisition module is used to obtain the hole area defect detection requirements of the mobile phone to be tested; A determination module, used to determine a detection environment for detecting the mobile phone to be tested based on the hole area defect detection requirement; A division module, used for dividing the area where the hole area of the mobile phone to be tested is located into a plurality of areas to be photographed; A shooting module, used for shooting the area to be shot based on the detection environment to obtain a shot image; The recognition module is used to recognize the plurality of captured images to obtain the defect type of the hole area.
9. A computing device comprising a memory, a processor, and a program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of a method for detecting defects in a hole area of a mobile phone as described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the steps of a method for detecting defects in a hole area of a mobile phone as described in any one of claims 1 to 7.
