Defect Detection Method, Device and Electronic Device Applied to Inner Wall Surface

Abstract

Embodiments of the present disclosure disclose a defect detection method, apparatus, and electronic device applied to the inner wall surface. A specific implementation of this method includes: locating connected holes based on a first image; for the hole information, performing the following processing steps: collecting inner wall images of the connected holes corresponding to the hole information; stitching each second image in the second image group; extracting inner wall features from the stitched image; determining the hole type of the connected hole corresponding to the hole information according to the inner wall features; in response to the hole type being a threaded hole type, generating hole detection information through a first defect detection model and the inner wall features; in response to the hole type being a non-threaded hole type, generating hole detection information through a second defect detection model and the inner wall features. This implementation realizes low-cost defect detection for the inner wall of components while ensuring detection efficiency and accuracy.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly to a defect detection method, apparatus, and electronic device applied to the inner wall surface. Background Art

[0002] In the manufacture of automotive components, component manufacturing is usually carried out by a numerical control machine tool using methods such as cutting and drilling. Especially for the holes contained in the components, when there are defects in the holes, it seriously affects the subsequent assembly accuracy. Currently, when detecting the inner wall surface of the holes, the following methods are usually used: (1) Magnetic particle testing method (2) Eddy current testing method (3) Ultrasonic testing method.

[0003] However, when using the above methods, the following technical problems often exist:

[0004] First, the magnetic particle testing method is only applicable to the surface defect detection of ferromagnetic materials, and it is difficult to effectively detect the surface defects of non-ferromagnetic materials;

[0005] Second, the eddy current testing method is applicable to conductive materials or non-metallic materials, with limited applicable materials, and at the same time, it cannot effectively display the type and size of defects;

[0006] Third, the ultrasonic testing method is not applicable to components with complex structures, and at the same time, the detection efficiency is low;

[0007] The above information disclosed in this background art section is only used to enhance the understanding of the background of the inventive concept, and thus, it may include information that does not form the prior art known to those of ordinary skill in the art. Summary of the Invention

[0008] This content part of the present disclosure is used to introduce the inventive concept in a brief form, and these inventive concepts will be described in detail in the subsequent detailed implementation part. This content part of the present disclosure is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0009] Some embodiments of the present disclosure propose a defect detection method, apparatus, and electronic device applied to the inner wall surface to solve one or more of the technical problems mentioned in the above background art section.

[0010] In a first aspect, some embodiments of the present disclosure provide a method for defect detection applied to an inner wall surface. The method includes: collecting a first image, where the first image is a front top view image of a component to be detected, and the component to be detected includes at least one connected hole; locating the connected holes based on the first image to obtain a set of hole information, where the hole information in the set of hole information includes: hole inner diameter and hole center coordinates; for each piece of hole information in the set of hole information, perform the following processing steps: collect an inner wall image of the connected hole corresponding to the hole information according to the hole inner diameter and hole center coordinates included in the hole information to obtain a second image group; splice the respective second images in the second image group to obtain a spliced image; extract inner wall features from the spliced image to generate inner wall features; determine the hole type of the connected hole corresponding to the hole information according to the inner wall features, where the hole type includes: threaded hole type and non-threaded hole type; in response to the hole type being the threaded hole type, generate hole detection information through a first defect detection model and the inner wall features, where the hole detection information characterizes whether there is a defect in the connected hole corresponding to the hole information and the defect description when there is a defect; in response to the hole type being the non-threaded hole type, generate the hole detection information through a second defect detection model and the inner wall features; determine whether the component to be detected is qualified according to the obtained set of hole detection information; in response to being unqualified, move the component to be detected to a candidate area.

[0011] Second aspect, some embodiments of the present disclosure provide a defect detection device applied to the inner wall surface. The device includes: an acquisition unit configured to acquire a first image, where the first image is a top-down view image of a component to be detected, and the component to be detected includes at least one communicating hole; a communicating hole positioning unit configured to perform communicating hole positioning based on the first image to obtain a set of hole information, where the hole information in the set of hole information includes: hole inner diameter and hole center coordinates; an execution unit configured to, for each piece of hole information in the set of hole information, perform the following processing steps: acquire an inner wall image of the communicating hole corresponding to the hole information based on the hole inner diameter and hole center coordinates included in the hole information to obtain a second image group; splice the respective second images in the second image group to obtain a spliced image; extract inner wall features from the spliced image to generate inner wall features; determine the hole type of the communicating hole corresponding to the hole information based on the inner wall features, where the hole type includes: threaded hole type and non-threaded hole type; in response to the hole type being the threaded hole type, generate hole detection information through a first defect detection model and the inner wall features, where the hole detection information characterizes whether there is a defect in the communicating hole corresponding to the hole information and the defect description when there is a defect; in response to the hole type being the non-threaded hole type, generate the hole detection information through a second defect detection model and the inner wall features; a determination unit configured to determine whether the component to be detected is qualified based on the obtained set of hole detection information; a moving unit configured to move the component to be detected to a candidate area in response to being unqualified.

[0012] Third aspect, some embodiments of the present disclosure provide an electronic device, including: one or more processors; a storage device storing one or more programs thereon, when the one or more programs are executed by the one or more processors, enabling the one or more processors to implement the method described in any implementation manner of the first aspect.

[0013] Fourth aspect, some embodiments of the present disclosure provide a computer-readable medium storing a computer program thereon, where the program, when executed by a processor, implements the method described in any implementation manner of the first aspect.

[0014] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the defect detection method applied to the inner wall surface in some embodiments of the present disclosure, high-accuracy defect detection for the inner wall surface of components is achieved in combination with images. Specifically, the reasons for the low accuracy of defect detection are as follows: First, the magnetic particle detection method is only applicable to the surface defect detection of ferromagnetic materials, and it is difficult to effectively detect surface defects of non-ferromagnetic materials; Second, the eddy current detection method is applicable to conductive materials or non-metallic materials, with limited applicable materials, and at the same time, it cannot effectively display the type and size of defects; Third, the ultrasonic detection method is not applicable to components with complex structures, and the detection efficiency is low. Based on this, the defect detection method applied to the inner wall surface in some embodiments of the present disclosure performs defect detection by combining images, with low implementation costs and reduced material limitations for components by different detection methods (magnetic particle detection method, eddy current detection method). Specifically, first, a first image is collected, where the first image is a front top view image of the component to be detected, and the component to be detected includes at least one connected hole. Secondly, based on the first image, the connected holes are located to obtain a hole information set, where the hole information in the hole information set includes: hole inner diameter and hole center coordinates. In this way, the positioning of the holes to be detected is achieved. Then, for each hole information in the hole information set, the following processing steps are performed: The first step is to collect images of the inner wall of the connected hole corresponding to the hole information according to the hole inner diameter and hole center coordinates included in the hole information to obtain a second image group; in this way, the image collection of the hole inner wall is achieved. The second step is to splice the individual second images in the second image group to obtain a spliced image; the third step is to extract the inner wall features of the spliced image to generate inner wall features; the fourth step is to determine the hole type of the connected hole corresponding to the hole information based on the inner wall features, where the hole type includes: threaded hole type and non-threaded hole type; in this way, the hole type is determined. In practice, different hole types correspond to different hole characteristics, and at the same time, the corresponding hole defect types are also different. The fifth step is to generate hole detection information through the first defect detection model and the inner wall features in response to the hole type being the threaded hole type, where the hole detection information characterizes whether there are defects in the connected hole corresponding to the hole information and the defect description when there are defects; the sixth step is to generate the hole detection information through the second defect detection model and the inner wall features in response to the hole type being the non-threaded hole type. Further, based on the obtained hole detection information set, it is determined whether the component to be detected is qualified. Finally, in response to being unqualified, the component to be detected is moved to the candidate area. In this way, low-cost defect detection for the inner wall of components is achieved while ensuring detection efficiency and accuracy. Brief Description of the Drawings

[0015] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and the elements and elements are not necessarily drawn to scale.

[0016] Figure 1 is a flowchart of some embodiments of a defect detection method applied to the inner wall surface according to the present disclosure;

[0017] Figure 2 is a schematic diagram of the component style of the component to be detected;

[0018] Figure 3 is a schematic structural diagram of some embodiments of a defect detection device applied to the inner wall surface according to the present disclosure;

[0019] Figure 4 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. Specific Embodiments

[0020] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0021] In addition, it should be noted that for the sake of convenience of description, only parts related to the relevant invention are shown in the drawings. Without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0022] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependent relationships.

[0023] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0024] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0025] The present disclosure will be described in detail below with reference to the drawings and in combination with the embodiments.

[0026] Continue to refer to Figure 1 which shows the flow 100 of some embodiments of a defect detection method applied to an inner wall surface according to the present disclosure. The defect detection method applied to the inner wall surface includes the following steps:

[0027] Step 101, acquire a first image.

[0028] In some embodiments, the execution subject (e.g., a computing device) of the defect detection method applied to the inner wall surface can acquire the first image through a high-speed camera disposed above the conveyor belt. Among them, the first image is a front top view image of the component to be detected. The component to be detected includes at least one communicating hole. In practice, when there is a component to be detected moving to the image acquisition area corresponding to the high-speed camera, control the above high-speed camera to acquire the above first image.

[0029] As an example, taking Figure 2 the schematic diagram of the component style of the component to be detected shown, where the above component to be detected includes: 1 communicating hole 201 of the threaded hole type and 4 communicating holes 202 of the non-threaded hole type.

[0030] It should be noted that the above computing device can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster composed of multiple servers or terminal devices, or can be implemented as a single server or a single terminal device. When the computing device is embodied as software, it can be installed in the above-listed hardware devices. It can be implemented as, for example, multiple software or software modules for providing distributed services, or can be implemented as a single software or software module. No specific limitation is made here.

[0031] Step 102, perform communicating hole positioning according to the first image to obtain a hole information set.

[0032] In some embodiments, the above execution subject can perform communicating hole positioning according to the first image to obtain a hole information set. Among them, the hole information in the above hole information set includes: hole inner diameter and hole center coordinates. In practice, first, the above execution subject can determine the communicating holes in the first image through the HOG (Histogram of Oriented Gradient) algorithm. Then, the above execution subject can determine the hole center of the communicating hole and the hole distance from the hole center to the communicating hole as the hole information set. Among them, the hole inner diameter represents the hole radius of the communicating hole corresponding to the hole information.

[0033] As an example, further refer to Figure 2Schematic diagram of the component style of the component to be detected shown above. The above-mentioned execution entity can determine the hole inner diameter and hole center coordinates of the connected hole 201 of the threaded hole type as the corresponding hole information.

[0034] In some optional implementation manners of some embodiments, the above-mentioned execution entity performs connected hole positioning according to the above-mentioned first image to obtain a set of hole information, which may include the following steps:

[0035] First step, perform image binarization processing on the above-mentioned first image to generate a third image.

[0036] In practice, the above-mentioned execution entity can perform image binarization processing on the above-mentioned first image by the bimodal method to generate a third image.

[0037] Second step, determine the image difference corresponding to the above-mentioned third image and the background image to obtain a fourth image.

[0038] Among them, the above-mentioned background image corresponds to the same image acquisition position as the above-mentioned first image, and the above-mentioned background image is a conveyor belt image after binarization processing when the conveyor belt does not convey the component to be detected. In practice, the fourth image = the third image - the background image.

[0039] Third step, perform image cropping on the above-mentioned third image according to the region boundary included in the above-mentioned fourth image to obtain a fifth image.

[0040] In practice, after determining the image difference between the third image and the background image, therefore, a fourth image including the component boundary (region boundary) of the component to be detected can be obtained. Thus, the above-mentioned execution entity can use the component boundary of the component to be detected as the region boundary to perform image cropping on the above-mentioned third image to obtain a fifth image.

[0041] Fourth step, determine a set of hole boundaries through a pre-constructed hole detection model and the above-mentioned fifth image.

[0042] Among them, the hole boundary characterizes the boundary corresponding to the connected hole. The above hole detection model includes: a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, and a detection head. In practice, the first convolutional block, the second convolutional block, the third convolutional block, the fourth convolutional block, and the detection head are connected in series. The above first convolutional block includes a residual block A, a residual block B, and a feature superposition layer A. The input of the residual block B is the above residual block A. The input of the feature superposition layer A is the output of the above residual block B and the feature after performing a 1×1 convolution on the output of the residual module A. Among them, the network structures of the second convolutional block and the first convolutional block are the same. Specifically, the second convolutional block includes: a residual block C, a residual block D, and a feature superposition layer B. The third convolutional block includes: a residual block E, a residual block F1, a residual block F2, a residual block F2, a residual block F4, a residual block F5, and a feature superposition layer C. Among them, the residual block F1, the residual block F2, the residual block F2, the residual block F4, and the residual block F5 are connected in series. The residual block F1, the residual block F2, the residual block F2, the residual block F4, and the residual block F5 are equivalent to the residual block B. The network structures of the fourth convolutional block and the third convolutional layer are the same. Specifically, the fourth convolutional block includes: a residual block G, a residual block H1, a residual block H2, a residual block H3, a residual block H4, a residual block H5, and a feature superposition layer D. The detection head includes: a 1×1 convolutional layer, a pooling layer, and a fully connected layer.

[0043] In the fifth step, determine the central coordinates corresponding to the boundary center of each hole boundary in the above hole boundary set as the hole center coordinates, and determine the shortest distance from the boundary center of the above hole boundary to the above hole boundary as the hole inner diameter, so as to generate hole information and obtain the above hole information set.

[0044] Step 103: For each piece of hole information in the hole information set, perform the following processing steps:

[0045] Step 1031: According to the hole inner diameter and hole center coordinates included in the hole information, collect the inner wall images of the connected hole corresponding to the hole information to obtain a second image group.

[0046] In some embodiments, the above execution entity may collect the inner wall images of the connected hole corresponding to the hole information according to the hole inner diameter and hole center coordinates included in the hole information to obtain a second image group. Among them, the second image group may be the inner wall images of the connected hole corresponding to the hole information from different perspectives. Specifically, the above execution entity may control the camera to move to the hole center coordinates included in the hole information and rotatably collect the inner wall images of the connected hole corresponding to the hole information to obtain a second image group.

[0047] In some alternative implementations of some embodiments, the execution subject collects images of the inner wall of the connected holes corresponding to the hole information according to the hole inner diameter and hole center coordinates included in the hole information, and a second image group can be obtained, which may include the following steps:

[0048] First step, determine the hole perimeter according to the hole inner diameter included in the hole information.

[0049] Among them, the aperture perimeter = 2×π×hole inner diameter.

[0050] Second step, determine the ratio of the hole perimeter to the preset sector arc length to obtain the target ratio.

[0051] Among them, the above target ratio is greater than 0. In practice, in order to ensure that no image is missed when collecting images of the inner wall of the connected holes, therefore, the preset sector arc length can be set according to the image acquisition angle of the camera. In practice, the target ratio = hole perimeter / preset sector arc length.

[0052] Third step, in response to the target ratio being an integer, determine the target ratio as the image acquisition frequency.

[0053] Fourth step, in response to the target ratio being a non-integer, round up the target ratio to obtain the image acquisition frequency.

[0054] In practice, when the target ratio is an integer, the target ratio can be directly used as the image acquisition frequency. At this time, there is an overlapping boundary between every two adjacent second images in the second image group obtained by acquisition. When the target ratio is a non-integer, due to rounding up, the image acquisition frequency is obtained. At this time, there is an overlapping area between every two adjacent second images in the second images obtained by acquisition.

[0055] For example, if the target ratio is 3.1, the image acquisition frequency can be 4.

[0056] Fifth step, generate a set of acquisition surfaces according to the image acquisition frequency.

[0057] In practice, the sector arc length corresponding to the acquisition surface in the set of acquisition surfaces = 360 / image acquisition frequency.

[0058] Sixth step, according to the hole center coordinates included in the hole information, control the inner wall image acquisition device to collect images of the inner wall of the hole corresponding to each acquisition surface in the set of acquisition surfaces to generate a second image, and obtain the second image group.

[0059] Among them, the inner wall image acquisition device includes: a high-speed camera, an inclined light guide mirror, and a rotating shaft. The high-speed camera and the inclined light guide mirror are controlled by the rotating shaft to rotate in the same direction, and the rotating shaft is driven by a motor. In practice, the inclined light guide mirror is connected to the high-speed camera. When performing the second image acquisition, the inclined light guide mirror extends into the communication hole. Due to the reversibility of the optical path, the high-speed camera can collect the second image group. The inclined light guide mirror is a cylinder with an inclined surface, and the inclined surface faces the inner wall of the communication hole. In this way, for communication holes with a small hole inner diameter, the second image can be acquired by using inclined light guide mirrors of different sizes. Compared with the method of performing the second image acquisition on communication holes with different hole inner diameters by using high-speed cameras of different sizes, the implementation cost is greatly reduced.

[0060] Step 1032: Perform image stitching on each second image in the second image group to obtain a stitched image.

[0061] In some embodiments, the execution subject can perform image stitching on each second image in the second image group to obtain a stitched image.

[0062] In practice, the above-mentioned confidence in this contract can perform sequential image stitching on the second images in the second image group according to the image acquisition order of the second images to obtain the above-mentioned stitched image.

[0063] In some optional implementation manners of some embodiments, the execution subject performing image stitching on each second image in the second image group to obtain a stitched image may include the following steps:

[0064] First step: In response to the target ratio being equal to the image acquisition frequency, extract image boundary feature points from each second image in the second image group to obtain first boundary feature points.

[0065] In practice, the execution subject can use the SIFT (Scale Invariant Feature Transform) algorithm to extract image boundary feature points from the second image to obtain first boundary feature points. Among them, the first boundary feature points represent the set of feature points on the image boundary included in the second image. Specifically, the execution subject can only extract feature points from two image boundaries perpendicular to the rotation direction of the inclined light guide mirror.

[0066] Second step: According to the first boundary feature points corresponding to the second images, perform image stitching on every two adjacent images in the second image group to obtain the above-mentioned stitched image.

[0067] In practice, the above-mentioned execution entity can splice every two adjacent images in the above-mentioned second image group through feature point matching to obtain the above-mentioned spliced image.

[0068] Thirdly, in response to the fact that the above-mentioned target ratio is not equal to the above-mentioned image acquisition frequency, for each second image in the above-mentioned second image group, perform the following steps for extracting feature points in the image boundary region:

[0069] The first sub-step is to determine a first overlapping region and a second overlapping region.

[0070] Among them, the above-mentioned first overlapping region represents the overlapping region between the above-mentioned second image and the second image adjacent in the forward direction, and the above-mentioned second overlapping region represents the overlapping region between the above-mentioned second image and the second image adjacent in the reverse direction. In practice, since the image acquisition frequency and the set of acquisition surfaces are known, and at the same time, there is a second image corresponding to each acquisition surface, the overlapping region between every two adjacent second images can be determined. In practice, the first overlapping region and the second overlapping region are perpendicular to the rotation direction of the above-mentioned inclined surface light guide mirror.

[0071] As an example, the second image group includes: second image A, second image B, and second image C. Among them, second image A, second image B, and second image C are acquired clockwise according to the rotation direction of the inclined surface light guide mirror. For the second image B, the overlapping region between the second image B and the second image A is the second overlapping region. The overlapping region between the second image B and the second image C is the first overlapping region.

[0072] The second sub-step is to vertically divide the feature point acquisition lines for the above-mentioned first overlapping region and the above-mentioned second overlapping region at a preset interval to obtain a first set of feature point acquisition lines and a second set of feature point acquisition lines.

[0073] In practice, taking the first overlapping region as an example, if feature points of the entire region of the first overlapping region are acquired, the amount of data processing is large. At the same time, when performing image splicing subsequently, due to the increase in the number of feature points, the matching time consumption between feature points will also be further increased. Therefore, the present disclosure can discretize the first overlapping region into a first set of feature point acquisition lines in combination with the extraction method and splicing method of the first boundary feature points in the case where the target ratio is equal to the above-mentioned image acquisition frequency. At the same time, since the feature point acquisition lines of different overlapping regions are vertically divided at a preset interval, the problem of subsequent feature point mismatch can also be avoided.

[0074] The third sub-step is to extract boundary features for each first feature point acquisition line in the above-mentioned first set of feature point acquisition lines to generate second boundary feature points, thereby obtaining a second group of boundary feature points.

[0075] In practice, the above-mentioned execution entity can extract boundary features from the first feature point acquisition line through the SIFT algorithm to generate second boundary feature points.

[0076] The fourth sub-step is to extract boundary features from each second feature point acquisition line in the above-mentioned second feature point acquisition line set to generate third boundary feature points, obtaining a third boundary feature point group.

[0077] In practice, the above-mentioned execution entity can extract boundary features from the second feature point acquisition line through the SIFT algorithm to generate third boundary feature points.

[0078] The fourth step is to perform image stitching on every two adjacent images in the above-mentioned second image group according to the second boundary feature point group and the third boundary feature point group corresponding to the second image, obtaining the above-mentioned stitched image.

[0079] In practice, for the corresponding first boundary point feature point group and third boundary feature point group, the feature point matching method can be adopted to perform image stitching on every two adjacent images in the above-mentioned second image group, obtaining the above-mentioned stitched image.

[0080] The content of "in some optional implementation manners of some embodiments" as an inventive point of the present disclosure realizes accurate image stitching. Specifically, first, when the target ratio is equal to the above-mentioned image acquisition frequency, there is an overlapping boundary between two adjacent second images. Therefore, the stitching between two adjacent second images can be realized by extracting feature points from the boundary. Second, when the target ratio is not equal to the above-mentioned image acquisition frequency, there is an overlapping image area between two adjacent second images. To ensure accurate image stitching, it is often necessary to collect feature points for the entire image area (the first overlapping area and the second overlapping area). However, for the feature point collection of the entire image area, as the overlapping image area increases, the data processing amount increases accordingly. Therefore, the present disclosure first discretizes the overlapping area into feature point acquisition lines, and then collects feature points on the feature point acquisition lines, thereby greatly reducing the data processing amount and improving the processing speed. At the same time, since the same method, that is, discretizing the image area at a preset interval to obtain feature point acquisition lines, is adopted for different overlapping areas, during subsequent stitching, the second boundary feature points and third boundary feature points corresponding to the same feature point acquisition line can also be directly subjected to feature point matching, thereby further improving the feature point matching efficiency.

[0081] Step 1033 is to extract the inner wall features of the hole of the stitched image to generate inner wall features.

[0082] In some embodiments, the above-mentioned execution entity may extract the inner wall features of the spliced image to generate inner wall features. In practice, the above-mentioned execution entity may use a convolutional neural network model to extract the inner wall features of the spliced image to generate inner wall features.

[0083] In some optional implementation manners of some embodiments, the above-mentioned execution entity extracting the inner wall features of the above-mentioned spliced image to generate inner wall features may include the following steps:

[0084] First step, through the first image feature extraction network, perform initial inner wall feature extraction on the above-mentioned spliced image to obtain initial inner wall features.

[0085] Among them, the above-mentioned first image feature extraction network includes: 10 serially connected convolutional layers. In practice, the first image feature extraction network includes: convolutional layer A1, convolutional layer A2, convolutional layer A3, convolutional layer A4, convolutional layer A5, convolutional layer A6, convolutional layer A7, convolutional layer A8, convolutional layer A9, convolutional layer A10. The input of convolutional layer A1 is the above-mentioned spliced image. The output of convolutional layer A10 is the above-mentioned initial inner wall features.

[0086] Second step, generate the spliced image with enhanced brightness according to the image brightness enhancement network and the above-mentioned initial inner wall features.

[0087] Among them, the above-mentioned image brightness enhancement network includes: 10 image brightness enhancement blocks, and each image brightness enhancement block includes 1 convolutional block and 1 deconvolutional block.

[0088] Among them, the image brightness enhancement network includes: image brightness enhancement block A1, image brightness enhancement block A2, image brightness enhancement block A3, image brightness enhancement block A4, image brightness enhancement block A5, image brightness enhancement block A6, image brightness enhancement block A7, image brightness enhancement block A8, image brightness enhancement block A9, image brightness enhancement block A10. In practice, the output of image brightness enhancement block A10 is the above-mentioned spliced image with enhanced brightness. The input of image brightness enhancement block A10 is convolutional layer A1 and image brightness enhancement block A9. The input of image brightness enhancement block A9 is convolutional layer A2 and image brightness enhancement block A8. The input of image brightness enhancement block A8 is convolutional layer A3 and image brightness enhancement block A7. The input of image brightness enhancement block A7 is convolutional layer A4 and image brightness enhancement block A6. The input of image brightness enhancement block A6 is convolutional layer A5 and image brightness enhancement block A5. The input of image brightness enhancement block A5 is convolutional layer A6 and image brightness enhancement block A4. The input of image brightness enhancement block A4 is convolutional layer A7 and image brightness enhancement block A3. The input of image brightness enhancement block A3 is convolutional layer A8 and image brightness enhancement block A2. The input of image brightness enhancement block A2 is convolutional layer A9 and image brightness enhancement block A1. The input of image brightness enhancement block A1 is convolutional layer A10.

[0089] In the third step, through the second image feature extraction network, perform deep inner wall feature extraction on the above-mentioned spliced image after brightness enhancement to obtain the above-mentioned inner wall features.

[0090] Among them, the above-mentioned first image feature extraction network, the above-mentioned image brightness enhancement network, and the above-mentioned second image feature extraction network are included in the hole inner wall feature extraction model, and the above-mentioned second image feature extraction network includes 4 feature extraction blocks arranged in parallel and 1 feature fusion layer.

[0091] Among them, the second image feature extraction network includes: feature extraction block A1, feature extraction block A2, feature extraction block A3, and feature extraction block A4. The model structures of feature extraction block A1, feature extraction block A2, feature extraction block A3, and feature extraction block A4 are the same, and each includes 3 serially connected convolutional layers.

[0092] The content of the above "in some optional implementation manners of some embodiments" is another inventive point of the present disclosure, which realizes the extraction of the inner wall features of the inner wall image in the case of low brightness. Specifically, first, set up the first image feature extraction network to implement the initial image feature extraction. At the same time, the features output by each convolutional layer are brightened by the corresponding image brightness enhancement block. In this way, the extraction of the inner wall features of the inner wall image under brightness adjustment is realized.

[0093] Step 1034, determine the hole type of the hole corresponding to the hole information according to the inner wall features.

[0094] In some embodiments, the above-mentioned execution subject can determine the hole type of the hole corresponding to the hole information according to the inner wall features. Among them, the hole types include: threaded hole type and non-threaded hole type. In practice, the above-mentioned execution subject can input the inner wall features into the FC layer to obtain the hole type of the hole corresponding to the hole information. In practice, the FC layer can adopt a binary classifier. Specifically, as the hole types increase, a multi-classifier can also be selected to determine the hole type of the hole corresponding to the hole information.

[0095] Step 1035, in response to the hole type being the threaded hole type, generate hole detection information through the first defect detection model and the inner wall features.

[0096] In some embodiments, the above-mentioned execution entity may generate hole detection information based on the first defect detection model and the inner wall features in response to the hole type being a threaded hole type. Among them, the above-mentioned hole detection information characterizes whether there are defects in the connected holes corresponding to the above-mentioned hole information and the defect description when there are defects. In practice, when there are no defects in the connected holes corresponding to the hole information, the hole detection information is empty. When there are defects in the connected holes corresponding to the hole information, the hole detection information includes a set of defect description information, where the defect description information includes: defect type and defect location. In practice, the model structures of the first defect detection model and the second defect detection model are the same, but due to the different hole types being detected, that is, the corresponding defect types of the holes are also different, therefore, in the training stage, the above-mentioned execution entity can, based on the first defect detection model, train to obtain the second defect detection model through transfer learning. This can overcome the problem of insufficient model accuracy corresponding to small training samples. Specifically, the first defect detection model can adopt the MobileNet-SSD model as the backbone model of the first defect detection model and the second defect detection model. By selecting the MobileNet-SSD model, the processing speed is faster compared to models such as YOLO.

[0097] Step 1036, in response to the hole type being a non-threaded hole type, generate hole detection information based on the second defect detection model and the inner wall features.

[0098] In some embodiments, the above-mentioned execution entity generates hole detection information based on the second defect detection model and the inner wall features in response to the hole type being a non-threaded hole type.

[0099] Step 104, determine whether the component to be detected is qualified according to the obtained set of hole detection information.

[0100] In some embodiments, the above-mentioned execution entity may determine whether the component to be detected is qualified according to the obtained set of hole detection information. Among them, when the set of hole detection information is all empty, it indicates that the component to be detected is qualified. When there is non-empty hole detection information in the set of hole detection information, it indicates that the component to be detected is unqualified.

[0101] Step 105, in response to being unqualified, move the component to be detected to the candidate area.

[0102] In some embodiments, in response to non - compliance, the component to be detected is moved to a candidate area. Wherein, the candidate area can be an area where manual secondary detection of the component to be detected is carried out. In practice, although the image - based detection method has a relatively high detection accuracy, it still cannot guarantee 100% accurate component detection. In order to reduce the misjudgment rate, by setting up a candidate area, the component to be detected can be re - verified manually. Specifically, the above - mentioned execution entity can control the conveyor belt to turn, such as controlling a universal conveyor belt, to convey the component to be detected to the candidate area.

[0103] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the defect detection method applied to the inner wall surface in some embodiments of the present disclosure, high-accuracy defect detection for the inner wall surface of components is achieved in combination with images. Specifically, the reasons for the low accuracy of defect detection are as follows: First, the magnetic particle detection method is only applicable to the surface defect detection of ferromagnetic materials, and it is difficult to effectively detect surface defects of non-ferromagnetic materials; Second, the eddy current detection method is applicable to conductive materials or non-metallic materials, with limited applicable materials, and at the same time, it cannot effectively display the defect type and size; Third, the ultrasonic detection method is not applicable to components with complex structures, and the detection efficiency is low. Based on this, the defect detection method applied to the inner wall surface in some embodiments of the present disclosure performs defect detection by combining images, with low implementation costs, and at the same time reduces the material limitations of different detection methods (magnetic particle detection method, eddy current detection method) for components. Specifically, first, a first image is collected, where the first image is a front top view image of the component to be detected, and the component to be detected includes at least one communicating hole. Secondly, based on the first image, the communicating holes are located to obtain a hole information set, where the hole information in the hole information set includes: hole inner diameter and hole center coordinates. In this way, the positioning of the holes to be detected is achieved. Then, for each hole information in the hole information set, the following processing steps are performed: First step, according to the hole inner diameter and hole center coordinates included in the hole information, image acquisition of the inner wall of the communicating hole corresponding to the hole information is performed to obtain a second image group; In this way, image acquisition of the inner wall of the hole is achieved. Second step, image stitching is performed on each second image in the second image group to obtain a stitched image; Third step, inner wall feature extraction is performed on the stitched image to generate inner wall features; Fourth step, according to the inner wall features, the hole type of the communicating hole corresponding to the hole information is determined, where the hole type includes: threaded hole type and non-threaded hole type; In this way, the hole type is determined. In practice, different hole types correspond to different hole characteristics, and at the same time, the corresponding hole defect types are also different. Fifth step, in response to the hole type being the threaded hole type, through the first defect detection model and the inner wall features, hole detection information is generated, where the hole detection information characterizes whether there are defects in the communicating hole corresponding to the hole information and the defect description when there are defects; Sixth step, in response to the hole type being the non-threaded hole type, through the second defect detection model and the inner wall features, the hole detection information is generated. Further, according to the obtained hole detection information set, it is determined whether the component to be detected is qualified. Finally, in response to being unqualified, the component to be detected is moved to the candidate area. In this way, low-cost defect detection for the inner wall of components is achieved while ensuring detection efficiency and accuracy.

[0104] For further referenceFigure 3 , as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a defect detection device applied to the inner wall surface. These device embodiments correspond to Figure 1 the method embodiments shown, and the defect detection device applied to the inner wall surface can be specifically applied to various electronic devices.

[0105] As Figure 3 shown, a defect detection device 300 applied to the inner wall surface in some embodiments includes: an acquisition unit 301, a communication hole positioning unit 302, an execution unit 303, a determination unit 304, and a movement unit 305. Among them, the acquisition unit 301 is configured to acquire a first image, where the first image is a front top view image of a component to be detected, and the component to be detected includes: at least one communication hole; the communication hole positioning unit 302 is configured to perform communication hole positioning according to the first image to obtain a hole information set, where the hole information in the hole information set includes: hole inner diameter and hole center coordinates; the execution unit 303 is configured to, for each hole information in the hole information set, perform the following processing steps: according to the hole inner diameter and hole center coordinates included in the hole information, acquire an inner wall image of the communication hole corresponding to the hole information to obtain a second image group; splice the respective second images in the second image group to obtain a spliced image; extract inner wall features from the spliced image to generate inner wall features; according to the inner wall features, determine the hole type of the communication hole corresponding to the hole information, where the hole type includes: threaded hole type and non-threaded hole type; in response to the hole type being the threaded hole type, generate hole detection information through a first defect detection model and the inner wall features, where the hole detection information characterizes whether there is a defect in the communication hole corresponding to the hole information and the defect description when there is a defect; in response to the hole type being the non-threaded hole type, generate the hole detection information through a second defect detection model and the inner wall features; the determination unit 304 is configured to determine whether the component to be detected is qualified according to the obtained hole detection information set; the movement unit 305 is configured to move the component to be detected to a candidate area in response to being unqualified.

[0106] It can be understood that the various units described in the defect detection device 300 applied to the inner wall surface correspond to Figure 1 the respective steps in the method described with reference to. Therefore, the operations, features, and beneficial effects described above for the method also apply to the defect detection device 300 applied to the inner wall surface and the units included therein, and will not be elaborated here.

[0107] Next, with reference to Figure 4, which shows a schematic structural diagram of an electronic device (e.g., a computing device) 400 suitable for implementing some embodiments of the present disclosure. Figure 4 The illustrated electronic device is merely an example and should not impose any limitation on the functions and scope of use of the embodiments of the present disclosure.

[0108] As Figure 4 shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which may perform various appropriate actions and processes according to a program stored in the read-only memory 402 or a program loaded from the storage device 408 into the random access memory 403. In the random access memory 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the read-only memory 402, and the random access memory 403 are connected to each other via a bus 404. The input / output interface 405 is also connected to the bus 404.

[0109] Generally, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 4 the electronic device 400 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had. Figure 4 Each block shown in

[0110] may represent a device or, as needed, multiple devices.

[0111] It should be noted that the computer-readable media described in some embodiments of the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable signal medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0112] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (Hyper Text Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0113] The above computer-readable medium may be included in the above electronic device; or it may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by the electronic device, the electronic device is caused to: collect a first image, where the first image is a top-down view image of the component to be detected, and the component to be detected includes: at least one communicating hole; perform communicating hole positioning based on the first image to obtain a hole information set, where the hole information in the hole information set includes: hole inner diameter and hole center coordinates; for each hole information in the hole information set, perform the following processing steps: collect an inner wall image of the communicating hole corresponding to the hole information based on the hole inner diameter and hole center coordinates included in the hole information to obtain a second image group; splice the respective second images in the second image group to obtain a spliced image; extract inner wall features from the spliced image to generate inner wall features; determine the hole type of the communicating hole corresponding to the hole information based on the inner wall features, where the hole type includes: threaded hole type and non-threaded hole type; in response to the hole type being the threaded hole type, generate hole detection information through a first defect detection model and the inner wall features, where the hole detection information characterizes whether there is a defect in the communicating hole corresponding to the hole information and the defect description when there is a defect; in response to the hole type being the non-threaded hole type, generate the hole detection information through a second defect detection model and the inner wall features; determine whether the component to be detected is qualified based on the obtained hole detection information set; in response to being unqualified, move the component to be detected to a candidate area.

[0114] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++; and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0116] The units described in some embodiments of the present disclosure can be implemented in software or in hardware. The described units can also be provided in a processor. For example, it can be described as: a processor includes an acquisition unit, a communication hole positioning unit, an execution unit, a determination unit, and a movement unit. Among them, the names of these units do not constitute a limitation on the unit itself in some cases. For example, the determination unit can also be described as "the unit that determines whether the component to be detected is qualified according to the obtained set of hole detection information".

[0117] The functions described above can be at least partially performed by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Product (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), and so on.

[0118] The above description is only some preferred embodiments of the present disclosure and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features having similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A defect detection method applied to the inner wall surface, comprising: Collecting a first image, wherein the first image is a front top view image of a component to be detected, and the component to be detected includes: at least one connected hole; Locating the connected holes according to the first image to obtain a hole information set, wherein the hole information in the hole information set includes: hole inner diameter and hole center coordinates; For each hole information in the hole information set, perform the following processing steps: According to the hole inner diameter and hole center coordinates included in the hole information, collect inner wall images of the connected hole corresponding to the hole information to obtain a second image group; Stitching the second images in the second image group to obtain a stitched image; Extracting inner wall features from the stitched image to generate inner wall features; According to the inner wall features, determine the hole type of the connected hole corresponding to the hole information, wherein the hole type includes: threaded hole type and non-threaded hole type; In response to the hole type being the threaded hole type, generate hole detection information through a first defect detection model and the inner wall features, wherein the hole detection information characterizes whether there is a defect in the connected hole corresponding to the hole information and the defect description when there is a defect; In response to the hole type being the non-threaded hole type, generate the hole detection information through a second defect detection model and the inner wall features; Determine whether the component to be detected is qualified according to the obtained hole detection information set; In response to being unqualified, move the component to be detected to a candidate area.

2. The method according to claim 1, wherein, The locating the connected holes according to the first image to obtain a hole information set includes: Performing image binarization processing on the first image to generate a third image; Determining the image difference between the third image and the background image, to obtain a fourth image, wherein the background image corresponds to the same image acquisition position as the first image, and the background image is a conveyor belt image after binarization processing when the conveyor belt does not convey the component to be detected; Cropping the third image according to the region boundary included in the fourth image to obtain a fifth image; Determining a hole boundary set through a pre-constructed hole detection model and the fifth image, wherein the hole detection model includes: a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, and a detection head; Determining the central coordinate corresponding to the boundary center of each hole boundary in the hole boundary set as the hole center coordinate, and determining the shortest distance from the boundary center of the hole boundary to the hole boundary as the hole inner diameter, to generate hole information and obtain the hole information set.

3. The method according to claim 2, wherein The collecting inner wall images of the connected hole corresponding to the hole information according to the hole inner diameter and hole center coordinates included in the hole information to obtain a second image group includes: Determining the hole perimeter according to the hole inner diameter included in the hole information; Determining the ratio of the hole perimeter to a preset sector arc length to obtain a target ratio, wherein the target ratio is greater than 0; In response to the target ratio being an integer, determine the target ratio as the image acquisition frequency; In response to the target ratio being a non-integer, round up the target ratio to obtain the above-mentioned image acquisition frequency; Generate a set of acquisition planes according to the image acquisition frequency; According to the hole center coordinates included in the hole information, control the inner wall image acquisition device to perform image acquisition on the inner wall of the hole corresponding to each acquisition plane in the set of acquisition planes to generate a second image, and obtain the second image group. The inner wall image acquisition device includes: a high-speed camera, an inclined surface light guide mirror, and a rotating shaft. The high-speed camera and the inclined surface light guide mirror are controlled by the rotating shaft to rotate in the same direction, and the rotating shaft is driven by a motor.

4. The method according to claim 3, wherein The image stitching of each second image in the second image group to obtain a stitched image includes: In response to the target ratio being equal to the image acquisition frequency, extract image boundary feature points from each second image in the second image group to obtain first boundary feature points; Perform image stitching on every two adjacent images in the second image group according to the first boundary feature points corresponding to the second images to obtain the stitched image; In response to the target ratio not being equal to the image acquisition frequency, for each second image in the second image group, perform the following image boundary region feature point extraction steps: Determine a first overlapping region and a second overlapping region, where the first overlapping region represents the overlapping region between the second image and the second image adjacent in the forward direction, and the second overlapping region represents the overlapping region between the second image and the second image adjacent in the reverse direction; Vertically divide the feature point acquisition lines of the first overlapping region and the second overlapping region at a preset interval to obtain a first set of feature point acquisition lines and a second set of feature point acquisition lines; Extract boundary features from each first feature point acquisition line in the first set of feature point acquisition lines to generate second boundary feature points, and obtain a second set of boundary feature points; Extract boundary features from each second feature point acquisition line in the second set of feature point acquisition lines to generate third boundary feature points, and obtain a third set of boundary feature points; perform image stitching on every two adjacent images in the second image group according to the second set of boundary feature points and the third set of boundary feature points corresponding to the second images to obtain the stitched image.

5. The method according to claim 4, wherein The extraction of inner wall features of the hole from the stitched image to generate inner wall features includes: Perform initial inner wall feature extraction on the stitched image through a first image feature extraction network to obtain initial inner wall features, where the first image feature extraction network includes: 10 serially connected convolutional layers; Generate a brightness-enhanced stitched image according to an image brightness enhancement network and the initial inner wall features, where the image brightness enhancement network includes: 10 image brightness enhancement blocks, and an image brightness enhancement block includes 1 convolutional block and 1 deconvolutional block; Through the second image feature extraction network, perform deep inner wall feature extraction on the spliced image after brightness enhancement to obtain the inner wall features. Among them, the first image feature extraction network, the image brightness enhancement network, and the second image feature extraction network are included in the hole inner wall feature extraction model. The second image feature extraction network includes 4 parallelly arranged feature extraction blocks and 1 feature fusion layer.

6. A defect detection device applied to the inner wall surface, comprising: An acquisition unit configured to acquire a first image, where the first image is a front top view image of a component to be detected, and the component to be detected includes at least one communicating hole; A communicating hole positioning unit configured to perform communicating hole positioning based on the first image to obtain a set of hole information, where the hole information in the set of hole information includes: hole inner diameter and hole center coordinates; An execution unit configured to, for each piece of hole information in the set of hole information, perform the following processing steps: according to the hole inner diameter and hole center coordinates included in the hole information, acquire an inner wall image of the communicating hole corresponding to the hole information to obtain a second image group; perform image splicing on each second image in the second image group to obtain a spliced image; perform inner wall feature extraction on the spliced image to generate inner wall features; determine the hole type of the communicating hole corresponding to the hole information according to the inner wall features, where the hole type includes: threaded hole type and non-threaded hole type; in response to the hole type being the threaded hole type, generate hole detection information through the first defect detection model and the inner wall features, where the hole detection information characterizes whether there is a defect in the communicating hole corresponding to the hole information and the defect description when there is a defect; in response to the hole type being the non-threaded hole type, generate the hole detection information through the second defect detection model and the inner wall features; A determination unit configured to determine whether the component to be detected is qualified according to the obtained set of hole detection information; A moving unit configured to move the component to be detected to a candidate area in response to non-conformance.

7. An electronic device, comprising: One or more processors; A storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer-readable medium having a computer program stored thereon, wherein, The computer program, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images