Container airtightness detection method and device, electronic equipment and storage medium

Abstract

The application provides a container airtightness detection method, device, electronic equipment and storage medium, and belongs to the intelligent detection field, and comprises the following steps: determining the contour region of a target container in a to-be-detected product image; extracting a first sound source signal in the contour region; determining a weld position region in the contour region; extracting a second sound source signal in the weld position region from the first sound source signal; and positioning a possible leakage point on the target container based on the second sound source signal. The container airtightness detection method, device, electronic equipment and storage medium provided by the application creatively deeply fuse visual information and acoustic information, strictly limit the detected sound source signal in the weld position region where leakage is most likely to occur through two-stage progressive spatial filtering, thereby greatly eliminating the interference of environmental noise, signal reflection and non-key region signals, and realizing high-precision and high-reliability positioning of the container leakage point.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent detection, and in particular to a container airtightness detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] In the production process of household appliances, such as water heater inner tank, compressor and other pressure-bearing containers, there are very high airtightness requirements, which need to be detected before being taken off the production line. For the scene with relatively high airtightness detection requirements, a relatively accurate detection scheme is to carry out helium or hydrogen-nitrogen leakage detection. For occasions with relatively low airtightness detection requirements, microphone array detection technology is a relatively potential technology that can replace traditional water bubble detection. The principle of microphone array airtightness detection is to locate the ultrasonic signal generated by the gas leakage to locate the leakage source.

[0003] The current microphone array airtightness detection technology has the following problems: 1) The sound source signal generated by gas leakage will reflect between objects to produce false sound source signal, causing false alarm of the microphone array; 2) The quick sealing and air charging mechanism and pipeline during container detection will leak during use, causing interference with the microphone array detection of the real container leakage; 3) The current microphone array leakage detection and positioning technology can only locate the leakage within the dynamic range of the maximum leakage source, and cannot identify all the leakage sources with large differences in leakage degree, thus causing the problem of still existing leakage after subsequent repair.

[0004] Therefore, it is necessary to provide a new microphone array airtightness detection method to overcome the above problems. SUMMARY

[0005] The present application provides a container airtightness detection method, device, electronic equipment and storage medium to prevent false detection and missed detection in the current microphone array airtightness detection.

[0006] The present application provides a container airtightness detection method, which comprises the following steps:

[0007] Determine the contour area of the target container in the image of the product to be tested;

[0008] Extract the first sound source signal in the contour area;

[0009] Determine the weld position area in the contour area;

[0010] Extract the second sound source signal in the weld position area from the first sound source signal;

[0011] Based on the second sound source signal, locate the possible leakage point on the target container.

[0012] According to the container airtightness detection method provided by the application, the second sound source signal is used to locate a possible leakage point on the target container, which comprises the following steps:

[0013] An acoustic spectrum corresponding to the second sound source signal is obtained.

[0014] A signal intensity highest point on the acoustic spectrum is determined.

[0015] When the signal intensity highest point is determined to be valid, a leakage point of the target container is determined according to the position of the signal intensity highest point in the acoustic spectrum.

[0016] An influence range of the signal intensity highest point in the acoustic spectrum is shielded.

[0017] The determination of the signal intensity highest point on the acoustic spectrum is iteratively performed until the signal intensity highest point is determined to be invalid.

[0018] A set of leakage points of the target container determined in each iteration process is obtained.

[0019] The influence range of the signal intensity highest point refers to a connected region with all signal intensities greater than a preset signal intensity and centered on the signal intensity highest point.

[0020] According to the container airtightness detection method provided by the application, whether the signal intensity highest point is valid is determined by judging whether the signal-to-noise ratio of the signal intensity highest point is greater than a preset signal-to-noise ratio threshold and whether the influence range of the signal intensity highest point is greater than a preset minimum area threshold.

[0021] According to the container airtightness detection method provided by the application, the signal-to-noise ratio of the signal intensity highest point is determined based on the following steps:

[0022] The signal intensity of the signal intensity highest point is determined.

[0023] A noise evaluation region is determined, which is other regions in the acoustic spectrum after shielding the influence range of the signal intensity highest point.

[0024] The background noise intensity related to the noise evaluation region is determined.

[0025] Based on the signal intensity of the signal intensity highest point and the background noise intensity, the signal-to-noise ratio of the signal intensity highest point is determined.

[0026] According to the container airtightness detection method provided by the application, the determination of the contour region of the target container in the image of the product to be detected comprises the following steps:

[0027] convolve the to-be-tested product image in horizontal and vertical directions using a first-order differential operator to obtain a gradient amplitude and a gradient direction of each pixel point on the to-be-tested product image;

[0028] set a pixel point with a gradient amplitude greater than a preset high amplitude threshold as a strong edge point, and set a pixel point with a gradient amplitude less than or equal to the preset high amplitude threshold but greater than a preset low amplitude threshold as a weak edge point;

[0029] determine, based on the gradient direction of each weak edge point, part of the weak edge points connected to any strong edge point among all the weak edge points;

[0030] sequentially connect the strong edge points and the part of the weak edge points to construct a closed contour of the target container;

[0031] determine a contour region of the target container based on the closed contour.

[0032] According to the container airtightness detection method provided by the application, the method for determining the contour region of the target container in the to-be-tested product image further comprises:

[0033] determine a gradient amplitude graph of the to-be-tested product image;

[0034] perform binaryzation processing on the gradient amplitude graph to obtain a binaryzation gradient amplitude graph;

[0035] determine a closed contour of the target container from the binaryzation gradient amplitude graph;

[0036] determine a contour region of the target container based on the closed contour.

[0037] According to the container airtightness detection method provided by the application, the method for determining the contour region of the target container based on the closed contour comprises:

[0038] determine a target closed contour based on a contour area, a contour level and a contour position of each closed contour;

[0039] regard a region where the target closed contour is located as the contour region of the target container.

[0040] According to the container airtightness detection method provided by the application, the method for determining the contour region of the target container in the to-be-tested product image further comprises:

[0041] input the to-be-tested product image into a contour labeling model to obtain a contour region probability graph output by the contour labeling model;

[0042] The pixel points greater than the preset probability threshold and less than or equal to the preset probability threshold in the contour region probability graph are differentially valued to obtain a binary mask graph;

[0043] The contour region of the target container is determined based on the binary mask graph.

[0044] The contour labeling model is obtained by training based on product image samples, and each product image sample is pre-labeled with a contour region probability graph label.

[0045] According to the container airtightness detection method provided by the application, the first sound source signal in the contour region is extracted, which comprises:

[0046] A coordinate system transformation matrix between a microphone sound source coordinate system and a camera image coordinate system is determined, the microphone sound source coordinate system is a spatial coordinate system in which a microphone array for container airtightness detection is located, and the camera image coordinate system is a coordinate system in which a camera for collecting product images is located.

[0047] Based on the acoustic signals collected by the microphone array, a full-space acoustic map covering the container airtightness detection space is generated.

[0048] Each potential sound source position point in the full-space acoustic map is traversed.

[0049] The microphone sound source coordinates of each potential sound source position point in the microphone sound source coordinate system are projected to the camera image coordinate system by using the coordinate system transformation matrix to obtain corresponding image pixel coordinates.

[0050] It is judged whether the image pixel coordinates corresponding to each potential sound source position point are located in the contour region of the target container.

[0051] If the image pixel coordinates are located in the contour region, the region corresponding to the potential sound source position point in the full-space acoustic map is retained.

[0052] Based on all the retained regions in the full-space acoustic map, the first sound source signal is generated.

[0053] According to the container airtightness detection method provided by the application, the determination of the coordinate system transformation matrix between the microphone sound source coordinate system and the camera image coordinate system comprises:

[0054] A calibration object is selected, the calibration object is provided with a visual calibration pattern and an acoustic beacon, and the physical position relationship of the acoustic beacon relative to the visual calibration pattern is predetermined.

[0055] capturing the calibration object by the camera and identifying the visual calibration pattern to determine a pose of the visual calibration pattern in a camera image coordinate system;

[0056] acquiring acoustic signals emitted by the acoustic beacon by the microphone array to locate coordinates of the acoustic beacon in a microphone sound source coordinate system;

[0057] calculating the coordinate system transformation matrix based on the pose of the visual calibration pattern in the camera image coordinate system, the coordinates of the acoustic beacon in the microphone sound source coordinate system, and the physical position relationship.

[0058] According to the container airtightness detection method provided by the application, the determination of the weld position region in the contour region comprises:

[0059] acquiring a process design model of the target container;

[0060] acquiring a three-dimensional coordinate point sequence of all welds of the target container based on the process design model;

[0061] projecting the three-dimensional coordinate point sequence to the product image to be detected based on a pose transformation matrix between a model coordinate system of the process design model and a camera image coordinate system to acquire a weld line segment of all welds of the target container on the product image to be detected;

[0062] generating a weld mask on the product image to be detected based on the weld line segment;

[0063] determining an intersection of the weld mask and the contour region as the weld position region.

[0064] According to the container airtightness detection method provided by the application, the extraction of the second sound source signal in the weld position region from the first sound source signal comprises:

[0065] traversing each potential sound source position point in an acoustic spectrum corresponding to the first sound source signal;

[0066] projecting microphone sound source coordinates of each potential sound source position point in the microphone sound source coordinate system to the camera image coordinate system by using the coordinate system transformation matrix to obtain corresponding image pixel coordinates;

[0067] if the image pixel coordinates are located in the weld position region, retaining a region corresponding to the potential sound source position point in the acoustic spectrum corresponding to the first sound source signal;

[0068] generating the second sound source signal based on all retained regions in the acoustic spectrum corresponding to the first sound source signal.

[0069] According to the container airtightness detection method provided by the application, the target container is an inner container of an electric water heater.

[0070] According to the container airtightness detection method provided by the application, a solvent with rust prevention, strong tension and volatile characteristics is sprayed on the surface of the target container.

[0071] The application further provides a container airtightness detection device, comprising:

[0072] A container positioning unit is configured to determine a contour region of a target container in a product image to be detected.

[0073] A sound source signal preliminary screening unit is configured to extract a first sound source signal in the contour region.

[0074] A weld seam positioning unit is configured to determine a weld seam position region in the contour region.

[0075] A sound source signal secondary screening unit is configured to extract a second sound source signal in the weld seam position region from the first sound source signal.

[0076] A leakage point positioning unit is configured to position a possible leakage point on the target container based on the second sound source signal.

[0077] The application further provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the container airtightness detection method according to any one of the above when executing the program.

[0078] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the container airtightness detection method according to any one of the above.

[0079] The application further provides a computer program product comprising a computer program, wherein the computer program is executable on a processor to implement the container airtightness detection method according to any one of the above.

[0080] The container airtightness detection method, device, electronic device and storage medium provided by the application creatively combine visual information and acoustic information, strictly limit the detected sound source signal in the weld seam position region where leakage is most likely to occur through two-stage progressive spatial filtering, thereby greatly eliminating the interference of environmental noise, signal reflection and non-critical region signals, and achieving high-precision and high-reliability positioning of the container leakage point. BRIEF DESCRIPTION OF DRAWINGS

[0081] In order to make the technical solutions in the embodiments of the present application or the related art clearer, the accompanying drawings needed in the embodiments or the related art description will be briefly introduced. Obviously, the accompanying drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative effort based on these drawings.

[0082] Figure 1 is a flowchart of the container airtightness detection method provided by the present application.

[0083] Figure 2 is one of the flowcharts of locating the possible leakage points on the target container provided by the present application.

[0084] Figure 3 is another of the flowcharts of locating the possible leakage points on the target container provided by the present application.

[0085] Figure 4 is one of the flowcharts of determining the contour region of the target container in the image of the product to be tested provided by the present application.

[0086] Figure 5 is another of the flowcharts of determining the contour region of the target container in the image of the product to be tested provided by the present application.

[0087] Figure 6 is a third of the flowcharts of determining the contour region of the target container in the image of the product to be tested provided by the present application.

[0088] Figure 7 is a flowchart of extracting the first sound source signal in the contour region of the target container provided by the present application.

[0089] Figure 8 is a flowchart of determining the coordinate system transformation matrix between the microphone sound source coordinate system and the camera image coordinate system provided by the present application.

[0090] Figure 9 is a flowchart of determining the weld position region in the contour region provided by the present application.

[0091] Figure 10 is a flowchart of extracting the second sound source signal in the weld position region from the first sound source signal provided by the present application.

[0092] Figure 11 is a structural schematic diagram of the container airtightness detection device provided by the present application.

[0093] Figure 12 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0094] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the accompanying drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0095] It should be noted that, in the description of the present application, the terms "comprise", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitation, the element defined by the statement "comprising a" does not exclude the presence of another identical element in the process, method, article or equipment comprising the element.

[0096] The terms "first", "second" and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second" and the like are generally a class, and do not limit the number of objects, for example, the first object can be one or more.

[0097] The embodiments of the present application will be described below in conjunction with Figures 1-12 The container airtightness detection method, device, electronic equipment and storage medium provided by the present application are described.

[0098] Figure 1 The flowchart of the container airtightness detection method provided by the present application is shown in FIG. 1, which includes but is not limited to the following steps: Figure 1

[0099] Step 11, determining the contour region of the target container in the product image to be detected.

[0100] The product image to be detected refers to a digital image obtained by an image acquisition device (such as an industrial CCD or CMOS camera) shooting an entity product at a detection station on a production line. Specifically, an industrial camera (for example, a CMOS camera with a resolution of 1920x1080 pixels) can be used to shoot the surface to be detected of the product to be detected at a fixed station on the production line.

[0101] In order to ensure the image quality, the image acquisition device can be equipped with a ring-shaped LED light source to provide a uniform and shadow-free lighting environment, so as to enhance the contrast between the container contour and the background.​

[0102] The acquired images of the product under test are typically RGB three-channel color images. To reduce computational complexity and facilitate subsequent processing, the acquired color images can be converted into single-channel grayscale images using the following conversion formula: Gray = 0.299 R+0.587 G+0.114 B. Wherein, G, R, and B refer to the brightness values ​​of the red, green, and blue color channels of any pixel in the color image of the product under test, respectively, and Gray refers to the single-channel gray value of any pixel in the grayscale image after weighted summation.

[0103] Considering that dust and slight changes in lighting in the production environment of the product under test may introduce noise into the image of the product under test, a smoothing process can be performed on the grayscale image before contour region detection. This embodiment uses a Gaussian filter, for example, using a 5x5 or 7x7 Gaussian kernel to perform a convolution operation on the image, to effectively filter out Gaussian noise while preserving edge information as much as possible.

[0104] The target container refers to the object to be tested for airtightness, typically a pressure vessel with airtightness requirements. In a preferred embodiment of the present invention, the target container may be the inner tank of an electric water heater, the outer casing of an air conditioner compressor, or similar items found in household appliances.

[0105] The outline region refers to a two-dimensional planar region in the image of the product under test that can completely surround the main body of the target container and is separated from the background and surrounding environment (such as conveyor belts, clamps, etc.). It can be defined by one or more closed sets of pixel coordinates.

[0106] In step 11, machine vision technology can be used to separate the subject (target container) in the image of the product under test from the irrelevant background, thereby determining the outline area of ​​the target container as the key detection area. This step can be implemented based on a variety of image processing technologies.

[0107] For example, by analyzing the color, brightness, texture and other features of an image, edge detection algorithms, threshold segmentation algorithms, region growing algorithms or deep learning-based image segmentation models can be used to identify the boundary of the target container and then divide the contour region of the target container. The output form of the contour region can be a mask or a set of coordinates that defines the range occupied by the target container in the two-dimensional image space.

[0108] Step 12: Extract the first sound source signal within the contour area.

[0109] The first sound source signal refers to the sound signal collected by the acoustic acquisition device (such as a microphone array) after preliminary spatial filtering. The preliminary spatial filtering here refers to the signal obtained by filtering out the interference (such as air pipe leakage, wall reflected sound, etc.) from outside the target container after filtering out the interference from outside the target container on the basis of the full-space acoustic signal collected by the acoustic acquisition device.

[0110] The purpose of step 12 is to realize the first guidance and screening of visual information to acoustic information. Specifically, the spatial coordinate mapping relationship between the acoustic imaging system (such as a microphone array) and the visual imaging system (such as a camera) can be established in advance. After obtaining this spatial coordinate mapping relationship, any spatial sound source located by the acoustic system can be accurately projected onto the image taken by the camera. Therefore, the present application can realize the screening of the full-space acoustic signal by judging whether the projection position of the sound source falls within the determined contour region of the target container. All acoustic signals whose projection positions fall within the contour region are retained to form the first sound source signal.

[0111] Step 13, determining the weld position region in the contour region.

[0112] The weld is the part where different parts of the target container are connected during the manufacturing process of the target container, and according to process experience, it is the weakest link where leakage is most likely to occur. The weld position region refers to a two-dimensional planar region inside the contour region of the target container corresponding to the positions of all welds or specific key welds on the target container.

[0113] As an optional embodiment, after obtaining the contour region of the target container and performing the first spatial filtering to obtain the first sound source signal, the present application further narrows down the detection range and focuses on the most critical part of the target container air tightness detection. The determination of the weld position region can be achieved by identifying the specific model of the target container, and then retrieving the process design model (such as CAD drawing) of the product of this model from the pre-established process database. Since the process design model accurately records the geometric position information of all welds, by registering and aligning the geometric position information of the welds in the process design model with the real-time photographed product image, the weld position region can be accurately marked on the contour region.

[0114] Step 14, extracting the second sound source signal in the weld position region from the first sound source signal.

[0115] The second sound source signal is the final acoustic signal used for positioning obtained by performing a second spatial filtering on the first sound source signal.

[0116] Specifically, the present application determines whether each sound source component in the first sound source signal falls within the weld location region determined in the previous step. Only the sound source components falling within the weld location region are finally retained to form the second sound source signal. At this point, the second sound source signal obtained has maximally excluded the interference (such as friction sound of the shell connecting gap) possibly generated in the non-weld region, and has a very high signal-to-noise ratio.

[0117] Step 15, based on the second sound source signal, positioning the possible leakage point on the target container.

[0118] The possible leakage point on the target container refers to the final air tightness detection result, that is, one or more position point coordinates of gas leakage occurring in the weld location region of the target container.

[0119] When detecting the leakage point, the present application analyzes the second sound source signal obtained after the second spatial filtering, for example, finds the region with the most concentrated energy or the highest sound pressure level in the acoustic spectrum formed by the second sound source signal, and determines the center position of the region as the possible leakage point.

[0120] The container air tightness detection method provided by the present application first performs a first spatial filtering on the full-space acoustic signal using the contour region of the target container, effectively filtering out the noise interference from the external environment and the detection device itself; and then performs a second spatial filtering using the weld location region of the target container, accurately focusing the detection focus on the most critical leakage high-risk region. This double and progressive spatial filtering method can greatly improve the signal-to-noise ratio of the leakage signal, fundamentally solving the false alarm and missed alarm problems caused by signal reflection and environmental interference in the traditional microphone array detection technology, and significantly improving the accuracy and reliability of the container air tightness detection.

[0121] Figure 2 is one of the flowcharts for positioning the possible leakage point on the target container provided by the present application, as shown in Figure 2 The present application provides a method for positioning the possible leakage point on the target container based on the second sound source signal, mainly including but not limited to the following steps:

[0122] Obtaining the acoustic spectrum corresponding to the second sound source signal;

[0123] determining a signal intensity highest point on the acoustic map;

[0124] in a case where the signal intensity highest point is determined to be valid, determining a leak point of the target container according to a position of the signal intensity highest point in the acoustic map;

[0125] shielding an influence range of the signal intensity highest point in the acoustic map;

[0126] iteratively performing the determining of the signal intensity highest point on the acoustic map and the shielding of the influence range of the signal intensity highest point in the acoustic map until the signal intensity highest point is determined to be invalid;

[0127] obtaining a set of leak points of the target container determined in each iteration process;

[0128] The influence range of the signal intensity highest point refers to a connected region with all signal intensities greater than a preset signal intensity and centered on the signal intensity highest point.

[0129] The acoustic map is also called an acoustic camera image or a sound pressure level (SPL) distribution map, and is a two-dimensional or three-dimensional image that visualizes sound source information. Each pixel or voxel on the acoustic map represents a specific position in space, and the brightness, color or value thereof indicates the sound pressure level (SPL) or acoustic energy of the position. The acoustic map can be generated by processing the sound source signal through an acoustic positioning algorithm such as beamforming.

[0130] In the present application, the acoustic map corresponding to the second sound source signal is generated by calculating the second sound source signal obtained after twice spatial filtering using an acoustic positioning algorithm, and reflects only the sound source distribution in the weld position area.

[0131] Among all the pixel points (or voxel points) of the obtained acoustic map, the pixel point with the maximum sound pressure level or acoustic energy value corresponds to the most prominent sound source position in the current sound field in a physical sense, which is referred to as a signal intensity highest point in the present application. Therefore, the determination of the signal intensity highest point on the acoustic map can be a numerical optimization process, in which the coordinates of the vertex are found by traversing the numerical matrix of the acoustic map, and the coordinates are the most likely leak point.

[0132] After finding the point with the highest signal strength, it needs to be verified against the real situation to determine whether it is a leak signal or a noise spike. The effectiveness of the judgment can be based on a variety of criteria, such as the signal-to-noise ratio of the signal, the equal area of the signal coverage. Only in the case of judging its effectiveness, the position of the signal highest point in the acoustic map will appear, and through coordinate transformation, it will be formally recorded as a leak point on the target container.

[0133] In traditional acoustic detection, the sound field generated by a strong leak point often masks the sound field signals of surrounding weak leak points, resulting in only one bright spot being observed on the acoustic map, causing missed detection. In order to solve the problem that the traditional microphone array can only locate the leak point with the highest sound pressure level, and ignore other leak points with smaller leakage degree that may exist at the same time, the present application will shield the influence range of the signal strength highest point determined in the acoustic map. This influence range is specifically defined as: taking the just determined signal strength highest point as the center, all connected regions in the acoustic map with signal strength values greater than a preset signal strength. The shielding operation can be, for example, setting the signal strength values of all pixel points in this influence range to zero, or marking them as unusable in subsequent searches.

[0134] After shielding is completed, an iterative process as shown in Figure 2 will be started. The step of "determining the point with the highest signal strength in the acoustic map" will be performed again on the acoustic map that has been partially shielded, to find a new point with the highest signal strength. Then, it is judged again whether this new point with the highest signal strength is effective. If the new point with the highest signal strength is still effective, the above process is repeated: it is determined as a new leak point, and its corresponding influence range is shielded.

[0135] This iterative process will continue until, after a search, the signal strength highest point found is determined to be invalid (for example, its signal strength is lower than the preset effectiveness threshold, or in the unshielded area of the acoustic map, no effective signal point can be found). At this time, the iterative loop terminates.

[0136] Finally, the collection of all leak points determined in each effective iteration process is obtained, forming a complete report containing the positions of all identified leak points on the target container, thereby achieving comprehensive and accurate positioning of multiple possible leak points.

[0137] The container airtightness detection method provided by the application, through the innovative process of "searching-determining-shielding-iterating", after positioning a signal strength highest point, the influence range of the signal strength highest point is shielded actively, so that the sound field signal of a weaker leakage point which is originally shielded can be highlighted, and through iterative execution, all effective leakage points can be found, so that the masking effect in the technology is overcome, and the comprehensive and accurate positioning of multiple leakage points on the target container is realized.

[0138] When judging whether the signal strength highest point is effective, the application mainly determines whether the signal-to-noise ratio of the signal strength highest point is greater than a preset signal-to-noise ratio threshold and whether the influence range of the signal strength highest point is greater than a preset minimum area threshold. Based on this, the application provides an embodiment for positioning the leakage point that may exist on the target container, as shown in Figure 3 .

[0139] The target container is inflated to make the internal pressure of the target container reach a preset pressure threshold, so as to ensure that the target container reaches the working condition required for detection.

[0140] In the actual detection process, the image of the product to be detected and the original sound source data collected by the microphone array are obtained to provide basic data for subsequent analysis. For the obtained image of the product to be detected, the contour extraction technology is used to extract the image features, so as to obtain the contour region of the target container. Then, the contour region is used as a mask to perform the first spatial filtering on the full-space sound source signal collected by the microphone array, so as to obtain the first sound source signal.

[0141] The welding position region of the target container can be further positioned in combination with the model information of the target container, and then the welding position region is used as another mask to perform the first spatial filtering on the first sound source signal, so as to obtain the second sound source signal.

[0142] Next, the acoustic map corresponding to the second sound source signal is constructed, and the positioning of the leakage point with the highest signal strength on the acoustic map is performed. For each positioned signal strength highest point, a judgment is performed: the signal-to-noise ratio of the signal strength highest point is calculated, and the signal-to-noise ratio is compared with a preset signal-to-noise ratio threshold to determine whether the quality and definition of the signal meet the requirements; the influence range area of the signal strength highest point is evaluated, and the influence range area is compared with a preset minimum area threshold to determine whether the spatial characteristics of the signal meet the spatial distribution characteristics of the actual leakage point. Only when the above two conditions are met, the signal strength highest point is determined to be effective, and then the influence range of the signal strength highest point in the acoustic map is shielded to avoid the interference on the subsequent positioning.

[0143] The above positioning and shielding process is iterated until it is determined that the highest signal intensity point in the current atlas is no longer effective (i.e., the signal-to-noise ratio is lower than the signal-to-noise ratio threshold or the influence area is smaller than the preset minimum area threshold), at which time the iteration is stopped. Finally, the coordinate information of the leakage points obtained in each effective iteration is fused into the product image to be tested to realize spatial fusion labeling of the leakage points on the target container, and all potential leakage points of the target container are intuitively presented.

[0144] The container air tightness detection method provided by the application performs double determination of the signal-to-noise ratio and the area threshold for each screened highest signal intensity point, which can effectively improve the accuracy and reliability of the leakage point positioning, can not only reject false signals caused by noise or artifacts, but also ensure accurate capture of the spatial characteristics of actual leakage points, greatly improving the confidence and practical value of the air tightness detection result.

[0145] As an optional embodiment, the application provides a method for calculating the signal-to-noise ratio of the highest signal intensity point, mainly including but not limited to the following steps:

[0146] determining the signal intensity of the highest signal intensity point;

[0147] determining a noise evaluation region, the noise evaluation region being other regions in the acoustic atlas after shielding the influence range of the highest signal intensity point;

[0148] determining the background noise intensity related to the noise evaluation region;

[0149] based on the signal intensity of the highest signal intensity point and the background noise intensity, determining the signal-to-noise ratio of the highest signal intensity point.

[0150] In the iterative positioning process of the leakage point on the target container, after the highest signal intensity point in the acoustic atlas is positioned in each iteration process, the signal-to-noise ratio (SNR) thereof can be calculated to determine its effectiveness. The core of the signal-to-noise ratio is to quantify the ratio between the signal intensity (Signal) and the background noise intensity (Noise). In the application, the signal refers to the ultrasonic signal generated due to leakage, and the noise refers to the background ultrasonic signal generated by the environment, the device itself, etc. when there is no leakage.

[0151] The highest signal intensity point found on the acoustic atlas is denoted as L_max, and the sound pressure level value SPL_max of the highest signal intensity point is defined as the signal intensity S, for example, S=SPL_max=75dB.

[0152] The key to calculating the signal-to-noise ratio (SNR) is to avoid including the signal itself in the background noise. Therefore, before calculating the background noise intensity, the influence range of the highest signal intensity point L_max is determined. For example, the search is expanded outward from the highest signal intensity point L_max, and all connected regions adjacent to the highest signal intensity point L_max and whose sound pressure level is higher than a certain relative threshold (e.g., SPL_max - 6dB) are considered as the influence range of the highest signal intensity point.

[0153] The noise assessment area can be defined as the area within the weld location of the target vessel, excluding all other areas within the influence range of the highest signal strength point L_max. This ensures that the assessment is of the real background noise that exists simultaneously with the leakage signal.

[0154] By traversing all pixels (or voxels) within the defined noise assessment area and reading their sound pressure level values, the average of these sound pressure level values ​​can be calculated to obtain the background noise intensity corresponding to the noise assessment area.

[0155] Considering that the unit of measurement for sound pressure level is decibels (dB), directly averaging decibel values ​​is physically inaccurate because decibels are logarithmic units. Therefore, after obtaining the sound pressure level values ​​of all pixels within the noise assessment area, this invention performs the following operations to obtain the background noise intensity:

[0156] Each pixel within the noise evaluation area i The sound pressure level value SPL_i is converted back to the sound intensity (or sound power value) I_i on a linear scale using the following formula: I_i = 10^(SPL_i / 10).

[0157] By calculating the arithmetic mean of all these linear sound intensities I_i, the average sound intensity I_noise_avg can be obtained. Then, converting this average sound intensity I_noise_avg back to decibels (dB) gives the final background noise intensity N, with the formula: N = SPL_noise = 10. log10(I_noise_avg).

[0158] Signal-to-noise ratio (SNR) in decibels is simply the difference between signal strength S and noise strength N:

[0159] SNR(dB)=SN=SPL_max-SPL_noise.

[0160] SPL_noise refers to the background noise intensity.

[0161] Finally, the calculated SNR value is compared with a preset SNR threshold T_snr (e.g. 5dB), and the comparison result can be used as a basis for judging the effectiveness of the signal strength peak.

[0162] Next, the outline region of the target container is determined in the container airtightness detection method provided by the present application through specific embodiments.

[0163] As an optional implementation, the first method for determining the outline region of the target container in the product image provided by the present application is a gradient-based edge detection and edge connection technology-based outline region method, as shown in the following figure, mainly including but not limited to the following steps: Figure 4

[0164] Step 111: using a first-order differential operator to convolve the product image in horizontal and vertical directions to obtain the gradient amplitude and gradient direction of each pixel point on the product image.

[0165] The first-order differential operator (such as Sobel operator) is used to perform convolution operation on the horizontal direction and vertical direction of the product image. This operation can calculate the gradient value of each pixel point in the horizontal direction and vertical direction of the product image.

[0166] Subsequently, based on the gradient values, the gradient amplitude (edge strength) of each pixel point is calculated by the Pythagorean theorem, that is, the vector sum of the horizontal direction and vertical direction gradients. At the same time, the gradient direction (edge direction) of each pixel point is calculated, which is usually the angle between the gradient vector and the horizontal axis. The gradient amplitude reflects the strength of the edge, and the gradient direction indicates the direction of the edge, which lays a foundation for subsequent edge thinning and connection.

[0167] Step 112: setting the pixel point with a gradient amplitude greater than a preset high amplitude threshold as a strong edge point, and setting the pixel point with a gradient amplitude less than or equal to the preset high amplitude threshold but greater than a preset low amplitude threshold as a weak edge point.

[0168] By setting two amplitude thresholds, a preset high amplitude threshold and a preset low amplitude threshold, the gradient amplitudes calculated in step 111 are classified. All pixel points with amplitudes greater than the high threshold are classified as strong edge points, representing points with very obvious and reliable edge information; and pixel points with amplitudes between the low threshold and the high threshold are classified as weak edge points, representing points with weak edge strength or potential edge points. Points below the low threshold are generally considered as non-edge points and discarded. This classification method helps to accurately capture the outline edge and prevent important edge information from being missed.

[0169] ​Step 113, based on the gradient direction of each weak edge point, determine the part of weak edge points connected to any strong edge point in all the weak edge points.

[0170] For each weak edge point, determine whether its adjacent pixel point in the gradient direction is a strong edge point according to its gradient direction. Only when the weak edge point and at least one strong edge point constitute a connected relationship in space, the weak edge point is retained, otherwise it is excluded. This connection judgment based on the gradient direction allows to retain the continuous edge pixels along the edge direction, and effectively filters out the false edge points caused by independent isolation or noise, thereby optimizing the coherence of the edge map.

[0171] Step 114, sequentially connect the strong edge points and the part of weak edge points to construct the closed contour of the target container.

[0172] Sequentially connect all strong edge points and their connected effective weak edge points to form continuous edge lines. Through topological analysis to locate the start and end points, and using image processing techniques (such as edge tracking algorithm) to further close these edge lines, the closed contour of the target container is constructed. This step ensures that the container contour is complete and closed, providing a clear boundary for subsequent contour region definition.

[0173] Step 115, based on the closed contour, determine the contour region of the target container.

[0174] Based on the boundary of the above closed contour, the pixel region inside the contour is identified as the contour region of the target container, which accurately locates the position and range of the target container in the product image to be tested. The information of this contour region is used later to cooperate with the acoustic map to realize the spatial mapping of the corresponding sound source signal and the leakage positioning.

[0175] The container airtightness detection method provided by the present application greatly enhances the extraction accuracy and integrity of the target container contour edge through hierarchical edge detection and weak edge point screening based on gradient direction, reducing edge breakage and misjudgment caused by noise, interference and image blur. The closed contour region obtained provides accurate and reliable spatial information support for subsequent acoustic signal analysis, improves the spatial matching accuracy of multi-leakage point positioning in airtightness detection, and improves the overall stability and reliability of the detection system.

[0176] As another alternative embodiment, the second method for determining the contour region of the target container in the product image to be tested provided by the present application is a method based on gradient amplitude calculation and binarization to extract the closed contour and contour region of the container, as shown in Figure 5 The method mainly includes but is not limited to the following steps:

[0177] Step 121, determine the gradient amplitude map of the to-be-tested product image.

[0178] The image processing algorithm (such as Sobel operator, Prewitt operator or Roberts operator) is used to process the to-be-tested product image, and the gradient amplitude of each pixel point in the image is calculated. Each pixel value in the gradient amplitude map reflects the edge intensity at that position, which can highlight the significant features of the object boundary in the image. By calculating the gray level change rate of the image in the horizontal and vertical directions, a complete gradient amplitude map can be generated as the basis for subsequent edge detection.

[0179] Step 122, binarize the gradient amplitude map to obtain a binary gradient amplitude map.

[0180] In order to clearly distinguish the edge region from the non-edge region, the obtained gradient amplitude map can be subjected to binarization processing. The binarization process sets a suitable threshold value, and assigns a binary value "1" (indicating an edge) to the pixel points whose gradient amplitude is higher than the threshold value, and assigns a value "0" (indicating a non-edge) to the pixels whose gradient amplitude is lower than the threshold value. The threshold value can be dynamically adjusted according to the environmental light conditions and image quality to ensure that the edge features can be fully preserved while minimizing the influence of background noise.

[0181] Step 123, determine the closed contour of the target container from the binary gradient amplitude map.

[0182] Based on the binarized gradient amplitude map, the closed contour of the target container can be recognized and extracted through connected component analysis and morphological processing (such as dilation, erosion and closing operation). The specific operation includes finding the edge connected region in the image and filling the edge breakpoints to generate a closed curve. The closed contour accurately reflects the spatial shape of the edge of the target container, providing a reliable basis for positioning the edge of the target container.

[0183] Step 124, determine the contour region of the target container based on the closed contour.

[0184] Finally, according to the closed contour obtained in step 123, the pixel set corresponding to the inside of the closed contour is taken as the contour region of the target container. This contour region represents the spatial projection range of the target container in the to-be-tested product image.

[0185] The container airtightness detection method provided by the present application adopts gradient amplitude calculation and adaptive binarization technology, combined with morphological closed contour extraction, effectively solves the problem of insufficient or broken image edge details, ensures the integrity and accuracy of the closed contour of the target container, and has the advantages of simple operation, high calculation efficiency and strong adaptability. It can stably extract the contour region of the target container in various environments, significantly improve the matching degree and overall detection accuracy of signal spatial positioning in the airtightness detection process.

[0186] As an optional embodiment, the above-mentioned implementation step of determining the contour area of the target container based on the closed contour mainly includes but is not limited to:

[0187] determining a target closed contour based on the contour area, the contour level and the contour position of each of the closed contours;

[0188] regarding the area where the target closed contour is located as the contour area of the target container.

[0189] The present application obtains a plurality of closed contour sets on the target container through an image processing procedure. For each closed contour, the contour area, the contour level structure and the position distribution in the image are calculated and analyzed.

[0190] The calculation of the contour area refers to measuring the number of pixels contained in the closed contour. The area size reflects the spatial range represented by the contour. Generally, the contour area of the target container needs to be within the expected size range to avoid selecting a noise contour that is too small or a background contour that is too large.

[0191] The contour level refers to the inclusion relationship between the contour and other contours, that is, whether the contour is an inner partition (such as a hole or a multi-layer structure) or an outer contour of a certain upper contour. According to the structural characteristics of the target container, the most suitable level (usually the outermost contour) is selected to ensure the integrity of the contour.

[0192] The contour position is determined by analyzing the spatial coordinate distribution of the contour, combining the preset container position range or model matching result, and screening the contour area that matches the overall structure position of the product to be tested, and excluding isolated or misaligned contours caused by environmental interference.

[0193] After determining the target closed contour that best matches the characteristics of the target container according to the above three indicators, the area surrounded by the determined target closed contour is regarded as the contour area of the target container, which serves as the spatial reference basis for subsequent air tightness detection and leakage positioning.

[0194] As another optional implementation, the present application provides a third method for determining the contour area of the target container in the image of the product to be tested, which is a target container contour area determination method based on a contour labeling model of deep learning, as shown in Figure 6 The method mainly includes but is not limited to the following steps:

[0195] Step 131: inputting the image of the product to be tested into the contour labeling model to obtain a contour area probability map output by the contour labeling model.

[0196] The product image to be tested is input into a pre-trained contour labeling model, which is trained based on product image samples, each of which is pre-labeled with a contour region probability map label. The model learns how to accurately calculate the contour distribution probability from the input image.

[0197] After training, the contour labeling model can output a contour region probability map with the same resolution as the input image. Each pixel value in the contour region probability map represents the probability of the pixel belonging to the container contour region, with a value range of [0, 1]. By comparing the output probability map of the contour labeling model with the manually labeled contour region probability map label, the loss between the two is calculated. Common loss functions include binary cross-entropy loss or Dice loss. Using optimization algorithms such as Adam, the weights in the contour labeling model are updated based on the calculated loss, with the goal of minimizing the value of the loss function. When the performance of the trained contour labeling model reaches the preset standard (such as a segmentation accuracy of over 99.5%) on the validation set, training is stopped, and the trained model weight file is saved for subsequent online deployment.

[0198] Among them, the contour labeling model can be selected from Mask R-CNN (a model for instance segmentation) or other segmentation models such as the DeepLab series.

[0199] Step 132, differentially assigning pixels in the contour region probability map that are greater than a preset probability threshold and less than or equal to the preset probability threshold to obtain a binary mask map.

[0200] For the contour region probability map, a preset probability threshold (e.g., 0.5) can be set for binary processing. Specifically, pixels with a contour probability greater than the preset threshold are assigned a value of "1", indicating that the pixel is determined to be an effective part of the container contour region; pixels with a probability less than or equal to the preset threshold are assigned a value of "0", indicating a non-contour region. Through this differential assignment, a clear binary mask map is generated, effectively separating the contour from the background.

[0201] Step 133, determining the contour region of the target container based on the binary mask map.

[0202] Based on the obtained binary mask map, morphological processing (such as opening operation, closing operation, etc.) can be further used to optimize the mask boundary and remove noise or small area isolated regions. Finally, the continuous and closed region in the mask is determined as the contour region of the target container, providing an accurate spatial basis for subsequent air tightness detection and leakage positioning.

[0203] The container airtightness detection method provided by the application adopts a deep learning contour labeling model to realize automatic contour region extraction, has stronger adaptability and robustness than a traditional gradient-based edge detection method, and can effectively cope with the influence of complex background, noise interference and image quality changes. Through probability graph threshold segmentation and morphological optimization, the extracted contour region is accurate and complete, which helps to improve the accuracy and reliability of target container positioning in airtightness detection.

[0204] One of the core purposes of the application is to eliminate the influence of interference sound sources generated by the external environment of the target container (such as air filling pipelines, clamps, wall reflections, etc.) on the detection results. To this end, the application proposes a spatial filtering method, the essence of which is to create a virtual mask defined by the visual system of the camera, which only allows sound source signals located in the contour region of the target container to pass through the processing, and screens or ignores all sound source signals from outside the contour region.

[0205] The following will be described in detail in conjunction with related embodiments, and the application is how to specifically screen out the first sound source signal from the full-space acoustic signal collected by the microphone array.

[0206] Figure 7 is the flowchart of extracting the first sound source signal in the contour region of the target container provided by the application, as Figure 7 shown, mainly includes but is not limited to the following steps:

[0207] Step 211, determining the coordinate system transformation matrix between the microphone sound source coordinate system and the camera image coordinate system; the microphone sound source coordinate system is the spatial coordinate system in which the microphone array for container airtightness detection is located, and the camera image coordinate system is the coordinate system in which the camera for collecting the image of the product to be tested is located.

[0208] To realize the extraction of the contour region of the target container, it is necessary to establish the unity of the visual coordinate system (i.e. the camera image coordinate system) and the acoustic coordinate system (i.e. the microphone sound source coordinate system), and the key is to determine the coordinate system transformation matrix connecting the two different spatial coordinate systems.

[0209] The space where the microphone array is located is called the microphone sound source coordinate system, which is the physical three-dimensional spatial coordinate system of acoustic signal collection; the space where the camera is located is called the camera image coordinate system, which corresponds to the two-dimensional pixel coordinate system of the product to be tested image. Through the calibration process, for example, using a known reference object or a made calibration object for space calibration, the coordinate system transformation matrix (including a rotation matrix and a translation vector) between the two is calculated to realize accurate coordinate conversion from the sound source space point to the image pixel point.

[0210] Step 212, generating a full-space acoustic graph covering the container airtightness detection space based on the acoustic signal collected by the microphone array.

[0211] The acoustic signals collected by the microphone array are reconstructed into a full-space acoustic map covering the detection space by a sound source localization algorithm (e.g. delay-and-sum beamforming, acoustic imaging algorithm, etc.). The acoustic map represents the sound intensity distribution at each potential sound source location point in the space around the target container, and is the basis for the spatial distribution of the target container leakage signal.

[0212] Step 213, each potential sound source location point in the full-space acoustic map is traversed, and spatial mapping and judgment are prepared for each sound source point.

[0213] For each potential sound source location point traversed, the three-dimensional sound source coordinates in the microphone sound source coordinate system are converted and projected into the camera image coordinate system using the coordinate system transformation matrix that has been determined, to obtain the corresponding two-dimensional image pixel coordinates, so as to realize the spatial correspondence between the sound source location point and the container region in the visual image.

[0214] Step 214, using the coordinate system transformation matrix, the microphone sound source coordinates of each potential sound source location point in the microphone sound source coordinate system are projected into the camera image coordinate system to obtain the corresponding image pixel coordinates.

[0215] Step 215, it is judged whether the image pixel coordinates corresponding to each potential sound source location point are located within the contour region of the target container.

[0216] It is judged whether each converted image pixel coordinate falls within the predetermined contour region of the target container, which ensures that only potential sound source signals in the target container region are concerned, and noise sources or other irrelevant sound source signals outside the container are excluded.

[0217] Step 216, if the image pixel coordinates are located within the contour region, the region corresponding to the potential sound source location point in the full-space acoustic map is retained.

[0218] Only when it is determined that the pixel coordinates corresponding to a certain potential sound source location point are located within the contour region of the target container, the region corresponding to the potential sound source location point in the full-space acoustic map is retained. Otherwise, the sound signal corresponding to the potential sound source location point is discarded, so as to avoid mistakenly including sound sources outside the target container into the detection range.

[0219] Step 217, based on all the retained regions in the full-space acoustic map, the first sound source signal is generated.

[0220] Finally, based on all the retained potential sound source position points and their corresponding sound intensity values, a first sound source signal in the target container contour region is synthesized. The first sound source signal can more accurately reflect the acoustic characteristics of the target container interior and contour region, providing accurate data basis for subsequent air tightness detection and leakage point positioning.

[0221] The application provides acoustic spectrum optimization screening based on coordinate system conversion and space mapping, realizes high fusion of container image space information and acoustic signal space distribution, significantly improves the extraction accuracy of effective leakage sound sources in the target container contour region, effectively eliminates environmental noise sources, enhances the pertinence and reliability of air tightness detection, and improves the space matching degree and signal quality of the entire detection process.

[0222] As an optional embodiment, as shown in Figure 8 The application further provides a method for determining a coordinate system conversion matrix between a microphone sound source coordinate system and a camera image coordinate system, mainly including but not limited to:

[0223] Step 221, a calibration object is selected, and a visual calibration pattern and an acoustic beacon are arranged on the calibration object, and the physical position relationship of the acoustic beacon relative to the visual calibration pattern is predetermined.

[0224] When first installed and deployed or when key components (cameras, microphone arrays) are replaced, a one-time calibration procedure must be performed to establish an accurate mapping relationship between the two different sensor coordinate systems.

[0225] First, a calibration object can be made, and the dual-mode calibration object can be a planar calibration plate (such as a checkerboard or a circular visual calibration pattern), on which a plurality of acoustic beacons (such as piezoelectric ceramic buzzer pieces that can emit ultrasonic waves that can be located by a microphone array) are accurately installed at specific positions (for example, a center point or four corner points). The spatial position relationship (including relative distance and direction) between each acoustic beacon and the visual calibration pattern on the calibration object can be measured and determined in advance and stored as fixed physical parameters.

[0226] Step 222, the calibration object is photographed by the camera and the visual calibration pattern is identified to determine the pose of the visual calibration pattern in the camera image coordinate system.

[0227] The calibration object is placed in a typical working area of the target container, and the calibration object is photographed by the camera to obtain a to-be-tested product image containing the visual calibration pattern. Through computer vision technology, the visual calibration pattern corner points or feature points in the to-be-tested product image are identified, and the camera intrinsic and extrinsic parameters are calculated based on the feature points.

[0228] Camera intrinsic calibration is to take pictures of the calibration board from different angles, and calculate the camera's internal parameters (including focal length, principal point coordinates) and distortion coefficients by using the checkerboard or circular patterns on the board. This step corrects the lens distortion itself.

[0229] Camera extrinsic calibration is to calculate the camera's external parameters (rotation matrix R and translation vector T) relative to the world coordinate system (which can be defined as the plane where the calibration board is located) according to the position of the calibration board in the camera's field of view.

[0230] Finally, the three-dimensional pose (including position and orientation) of the visual calibration pattern in the camera image coordinate system can be determined. This process can be completed using traditional PnP (Perspective-n-Point) algorithm or deep learning assisted visual positioning algorithm.

[0231] Step 223: Collect the acoustic signals emitted by the acoustic beacons through the microphone array, and locate the coordinates of the acoustic beacons in the microphone sound source coordinate system.

[0232] At the same location, collect the acoustic signals emitted by each acoustic beacon through the microphone array. Use acoustic positioning algorithms (such as sound source positioning, beamforming, time difference method, etc.) to calculate the accurate three-dimensional spatial coordinates of the acoustic beacons in the microphone sound source coordinate system.

[0233] Step 224: Based on the pose of the visual calibration pattern in the camera image coordinate system, the coordinates of the acoustic beacons in the microphone sound source coordinate system, and the physical position relationship, calculate the coordinate system transformation matrix.

[0234] After the above steps, several pairs of coordinate points (depending on the number of acoustic beacons) can be obtained: the known physical position of the acoustic beacon in the world coordinate system (x_world, y_world, z_world) and the positioning result of the acoustic beacon in the microphone array coordinate system (x_mic, y_mic, z_mic).

[0235] Through these pairs of coordinate points, the transformation matrix from the microphone array coordinate system to the world coordinate system can be solved. Since the camera's extrinsic parameters also establish its relationship with the world coordinate system, the final projection transformation relationship from the microphone sound source coordinate system to the camera image coordinate system can be obtained.

[0236] The present application realizes accurate fusion of the acoustic coordinate system and the visual coordinate system by using a visual calibration pattern and an acoustic beacon in combination with a physically measured spatial relationship. The dual-mode calibration method overcomes the shortcomings of single visual or acoustic calibration, significantly improves the accuracy and stability of the coordinate transformation matrix, provides a solid spatial registration basis for subsequent container leakage positioning based on acoustic signals, and effectively ensures the spatial mapping of the airtightness detection and the accuracy of acoustic analysis.

[0237] As another optional embodiment, the present application also provides a method for determining the coordinate transformation matrix between the microphone sound source coordinate system and the camera image coordinate system based on manually specified corresponding points. This method is particularly suitable for accurately mapping between the two coordinate systems by manual means when automatic calibration devices cannot be used or are inconvenient. The specific steps include but are not limited to:

[0238] A number of landmark points with clear physical features are selected on the surface of the target container, such as screw holes, container corner points, symbol marks, etc. These landmark points should be easily visually recognized and uniformly distributed on the structure of the target container to ensure the accuracy of spatial mapping.

[0239] The spatial three-dimensional coordinates of these landmark points in three-dimensional space are manually measured and recorded by the operator. The spatial three-dimensional coordinates are determined based on the microphone sound source coordinate system, i.e., relative to the spatial reference system of the microphone array. The measurement method can use laser ranging, mechanical scale or other high-precision measurement equipment to ensure the reliability of the coordinate position.

[0240] Subsequently, in the image of the product to be measured collected by the camera, the two-dimensional pixel coordinates corresponding to the above-mentioned landmark points are manually marked. The operator can complete the acquisition of the image coordinates by clicking on the corresponding feature points on the image through the graphical user interface.

[0241] After obtaining two sets of corresponding point sets (spatial three-dimensional coordinates and two-dimensional pixel coordinates), the perspective transformation matrix (or projection transformation matrix) can be solved using the corresponding point sets through mathematical methods. The perspective transformation matrix (which includes the internal and external parameter information of the camera and the spatial mapping relationship to form the coordinate transformation matrix) can be used to realize accurate projection conversion from the microphone sound source coordinate system to the camera image coordinate system.

[0242] The manual specification method of landmark points provided by the present application does not rely on complex calibration objects or automated equipment, is simple and flexible to operate, is suitable for rapid deployment in complex environments in actual detection sites, ensures high-precision mapping of the coordinate transformation by accurately selecting and measuring multiple spatial landmark points, and thus enhances the spatial fusion effect of acoustic signals and visual images in airtightness detection, and enhances the accuracy and practicality of detection.

[0243] Figure 9is a flowchart of determining the weld position area in the contour area provided by the present application, as shown in Figure 9 which mainly includes but is not limited to the following steps:

[0244] Step 311, obtaining the process design model of the target container.

[0245] The process design model is usually a three-dimensional CAD model or a digital twin model, which contains the accurate structural design and manufacturing process information of the target container. The present application provides several implementable ways to obtain the process design model of the target container.

[0246] In the production management of household appliances, a barcode or QR code containing the model information of each target container (such as a water heater liner) will be pasted at a fixed position. A special industrial code reader or a camera can be used to take high-definition images of the barcode area while taking images of the product to be tested. By calling mature barcode recognition libraries (such as ZBar, ZXing, etc.), the barcode in the image of the product to be tested can be decoded, and the model string of the target container (for example, "Model-A-Plus-2025") can be directly and accurately obtained.

[0247] In addition, the Optical Character Recognition (OCR) method can also be used to read the model of the target container, mainly for the case where the model of the target container has been printed on the surface by spraying, laser engraving, etc. First, locate the approximate area of the model text by template matching, etc., and then call the OCR engine (such as Tesseract OCR) to recognize the image in this area and extract the model string.

[0248] The image feature matching or deep learning classification method based on the appearance of the container can also be used to identify the model of the target container. For example, one or more standard images of each model of the container are taken, and the key feature points (such as using SIFT, SURF, etc.) are extracted to construct a feature library. In the actual detection process, the real-time image of the target container is matched with the key feature points of each model in the feature library, and the one with the highest matching degree is the model of the target container.

[0249] Prior to this, the process design model of each type of container is obtained in advance, usually a CAD (Computer Aided Design) file, which can be a 2D DXF file or a 3D STEP / IGES file. An offline processing program can be called to parse these CAD files and specifically extract the geometric information marked as welds (usually line segments or curves). The extracted weld information is stored in a database in a standardized format.

[0250] After obtaining the model string of the target container in the above manner, the process design model corresponding to the target container can be obtained by searching the above database using the determined model string as the query index.

[0251] Step 312, based on the process design model, obtaining a sequence of three-dimensional coordinate points of all welds of the target container.

[0252] Based on the retrieved process design model of the target container, the spatial position data of all welds can be obtained by calling the pre-defined weld information or through automatic recognition algorithms. The specific method includes:

[0253] If the process design model contains weld annotations, including relevant structural features (such as weld path, weld entity, etc.), the three-dimensional boundary points, center lines, or weld region boundary point sequences can be directly read.

[0254] If the model does not have explicit weld annotations, the weld region can be automatically identified through surface recognition (such as boundary detection, feature line extraction), and the corresponding three-dimensional coordinate point sequence can be fitted.

[0255] Step 313, according to the pose transformation matrix between the model coordinate system of the process design model and the camera image coordinate system, projecting the three-dimensional coordinate point sequence onto the image of the product to be tested to obtain the weld line segments of all welds of the target container on the image of the product to be tested.

[0256] The process design model of the target container exists in its own model coordinate system, while the collected image of the product to be tested corresponds to another camera image coordinate system. The spatial correspondence between the two can be achieved through a pre-computed pose transformation matrix (including a rotation matrix and a translation vector). In specific implementation, the following transformation is performed on each three-dimensional weld coordinate point:

[0257] The three-dimensional coordinate points are converted from the model coordinate system to the camera coordinate system space using the pose transformation matrix; and the three-dimensional coordinate points are mapped to the camera image coordinate system again through a camera projection model using the camera intrinsic parameters (focal length, principal point offset, distortion parameters), so as to obtain the projection position of the weld on the image of the product to be measured. After continuous mapping of all the weld points, a two-dimensional image line segment sequence corresponding to each weld of the target container can be formed.

[0258] In step 314, a weld mask is generated on the image of the product to be measured based on the weld line segment.

[0259] According to the projected weld line segment, a binary weld mask can be generated in the image of the product to be measured. The specific scheme includes: performing width expansion on the weld line segment, and setting the mask width according to the actual weld width and the image resolution, for example, expanding the line segment outward by a certain pixel range to form a belt-shaped area; using the dilation operation (Dilation) in image processing to realize the width expansion, so as to ensure that the weld mask area can completely cover the possible actual position of the weld and has a certain tolerance range; the mask image is a binary image, the pixels in the weld mask area are set to "1", and the background is "0".

[0260] In step 315, the intersection of the weld mask and the contour area is determined as the weld position area.

[0261] Finally, the intersection of the weld mask and the contour area of the target container obtained in advance can be calculated by using an image processing algorithm. The specific implementation can adopt the logical AND operation of the binary image, and the intersection area obtained is the part in the weld mask and within the contour area of the target container. Therefore, the intersection result is the finally determined weld position area, which ensures accurate weld positioning and does not exceed the container structure range, and provides a reliable spatial reference for subsequent acoustic detection and leakage analysis based on the weld area.

[0262] The container airtightness detection method provided by the application can accurately identify the weld position range in the actual product image by finely extracting the three-dimensional coordinates of the weld in the process design model and mapping them to a two-dimensional image using accurate spatial transformation, and generating a weld mask with appropriate width by image processing. At the same time, by the intersection of the weld mask and the contour of the target container, it is ensured that the identified weld area completely fits the container structure, effectively excluding the background and interference area, which significantly improves the accuracy and stability of the weld area positioning, and provides a reliable spatial basis for the key analysis and leakage source discrimination of the weld part in the airtightness detection.

[0263] Figure 10 is a flowchart of the application for extracting the second sound source signal in the weld position area from the first sound source signal, as shown in Figure 10 The main steps include but are not limited to the following steps:

[0264] Step 411, traverse each potential sound source position point in the acoustic map corresponding to the first sound source signal.

[0265] First, load the acoustic map corresponding to the first sound source signal, which is a three-dimensional spatial coordinate grid or point cloud structure, each coordinate point (potential sound source position point) carrying the corresponding sound intensity information. Through a loop traversal algorithm, from the starting spatial point to the terminal spatial point (for example, from the top left corner to the bottom right corner), access and read the three-dimensional coordinates and their corresponding sound energy intensity of each potential sound source position point one by one.

[0266] The traversal can use a spatial index structure such as Octree or KD-tree to accelerate the search, ensuring traversal efficiency while supporting fast access to large-scale acoustic map data.

[0267] Step 412, use the coordinate system transformation matrix to project the microphone sound source coordinates of each potential sound source position point in the microphone sound source coordinate system to the camera image coordinate system to obtain the corresponding image pixel coordinates.

[0268] For each potential sound source position point traversed, the coordinate system transformation matrix obtained in advance through calibration or measurement can be called to convert it from the microphone sound source coordinates to the two-dimensional camera image coordinates:

[0269] The rotation matrix and translation vector are used to complete the three-dimensional coordinate transformation from the microphone sound source coordinate system to the camera image coordinate system, obtaining the converted three-dimensional point. Combined with the camera intrinsic parameters (including focal length, principal point coordinates, radial and tangential distortion coefficients), the obtained three-dimensional point is projected to the two-dimensional image plane to obtain the accurate image pixel coordinates (x, y).

[0270] To reduce the influence of distortion, a camera distortion correction algorithm can also be used to correct the obtained image pixel coordinates to ensure the accuracy of the mapping.

[0271] Step 413, if the image pixel coordinates are located within the weld position region, the region corresponding to the potential sound source position point in the acoustic map corresponding to the first sound source signal is retained.

[0272] Using the pre-obtained weld mask (binary image), each projected pixel coordinate is judged for the region: according to the position of the image pixel coordinates in the weld mask, the corresponding pixel value is read, if the value is "1" (representing the weld mask region), it indicates that the potential sound source position point is inside the weld region.

[0273] Only the points meeting the condition retain their effective three-dimensional sound source data, and the effective sound source set of the weld area is constructed. Of course, a boundary tolerance can be set in the judgment process, for example, a tolerance range of ± several pixels near the weld mask boundary is allowed to cope with the uncertainty of the mask edge.

[0274] At step 414, the second sound source signal is generated based on all the retained regions in the acoustic map corresponding to the first sound source signal.

[0275] The sound intensity data in the sound source set in the weld position area obtained by the above screening is weighted or accumulated to generate the second sound source signal reflecting the weld position area. The sound intensity weighting can be combined with spatial neighborhood filtering, such as Gaussian filtering, to smooth the local sound intensity distribution and reduce the influence of random noise. A clustering algorithm (such as DBSCAN) can also be used to analyze the density of the selected sound source points to extract representative sound source distribution and further improve the spatial focusing degree of the signal.

[0276] The container airtightness detection method provided by the application effectively eliminates irrelevant sound source data outside the weld position area while retaining the real leakage sound source characteristics in the weld position area through specific space traversal, accurate coordinate transformation, and strict screening based on the weld mask, thereby providing a reliable and clear acoustic information basis for the airtightness detection system and facilitating the accurate implementation of subsequent leakage identification and positioning algorithms.

[0277] As an optional embodiment, the detection object of the container airtightness detection method provided by the application can be an electric water heater inner container or other sealed containers with airtightness detection requirements.

[0278] In actual detection, it is found that the signal-to-noise ratio of the ultrasonic signal generated by pure gas leakage is low, and water mist or soap water can amplify the ultrasonic signal intensity, but water vapor will affect the detection sensitivity of the microphone array unit. Therefore, the container airtightness detection method provided by the application uses a method of spraying a solvent with rust prevention, strong tension, and volatile characteristics on the surface of the target container to enhance the signal-to-noise ratio of the ultrasonic signal generated by the leakage point in the actual detection process.

[0279] As an optional embodiment, the solvent is a mixture of isopropyl alcohol, D-limonene, ethanol, glycerol, or a plurality of the above solvents.

[0280] Figure 11 Fig. 1 is a structural schematic diagram of a container airtightness detection device provided by the application, as shown in Fig. 1, the application further provides a container airtightness detection device, mainly including but not limited to: Figure 11

[0281] The container positioning unit 101 is used to determine the contour area of the target container in the image of the product to be detected.​

[0282] The sound source signal preliminary screening unit 102 is configured to extract a first sound source signal in the contour region;

[0283] The weld seam positioning unit 103 is configured to determine a weld seam position region in the contour region;

[0284] The sound source signal secondary screening unit 104 is configured to extract a second sound source signal in the weld seam position region from the first sound source signal;

[0285] The leakage point positioning unit 105 is configured to position a possible leakage point on the target container based on the second sound source signal.

[0286] It should be noted that the container airtightness detection device provided by the present application can realize the container airtightness detection method provided by any of the above embodiments when it is executed, and will not be described here.

[0287] The container airtightness detection device provided by the present application creatively deeply fuses visual information and acoustic information, strictly limits the detected sound source signal in the weld seam position region where leakage is most likely to occur through two-stage progressive spatial filtering, thereby greatly excluding the interference of environmental noise, signal reflection and non-key region signals, and realizing high-precision and high-reliability positioning of the container leakage point.

[0288] Figure 12 is a structural schematic diagram of an electronic device provided by the present application, as Figure 12 shown, the electronic device can include a processor (processor) 1210, a communication interface (communications interface) 1220, a memory (memory) 1230 and a communication bus 1240, wherein the processor 1210, the communication interface 1220, the memory 1230 complete the communication between each other through the communication bus 1240. The processor 1210 can call the logic instructions in the memory 1230 to execute the container airtightness detection method, which includes determining a contour region of a target container in a to-be-measured product image; extracting a first sound source signal in the contour region; determining a weld seam position region in the contour region; extracting a second sound source signal in the weld seam position region from the first sound source signal; and positioning a possible leakage point on the target container based on the second sound source signal.

[0289] In addition, the logic instructions in the memory 1230 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0290] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, and the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer can execute the container airtightness detection method provided by the above-mentioned embodiments, and the method comprises the following steps: determining a contour region of a target container in a to-be-tested product image; extracting a first sound source signal in the contour region; determining a weld position region in the contour region; extracting a second sound source signal in the weld position region from the first sound source signal; and positioning a possible leakage point on the target container based on the second sound source signal.

[0291] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, and the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer can execute the container airtightness detection method provided by the above-mentioned embodiments, and the method comprises the following steps: determining a contour region of a target container in a to-be-tested product image; extracting a first sound source signal in the contour region; determining a weld position region in the contour region; extracting a second sound source signal in the weld position region from the first sound source signal; and positioning a possible leakage point on the target container based on the second sound source signal.

[0292] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0293] Those skilled in the art can clearly understand the implementation of the embodiments by means of software and necessary general hardware platforms through the description of the above embodiments, and of course, the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0294] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of detecting the airtightness of a container, characterized in that, The method comprises the following steps: determining a contour region of a target container in a product image to be tested; extracting a first sound source signal in the contour region; determining a weld position region in the contour region; extracting a second sound source signal in the weld position region from the first sound source signal; locating a possible leakage point on the target container based on the second sound source signal; the step of locating a possible leakage point on the target container based on the second sound source signal comprises: obtaining an acoustic spectrum corresponding to the second sound source signal; determining a point of highest signal intensity on the acoustic spectrum; if the point of highest signal intensity is determined to be valid, determining a leakage point of the target container according to the position of the point of highest signal intensity in the acoustic spectrum; shielding the influence range of the point of highest signal intensity in the acoustic spectrum; iteratively performing the steps of determining the point of highest signal intensity on the acoustic spectrum and shielding the influence range of the point of highest signal intensity in the acoustic spectrum until the point of highest signal intensity is determined to be invalid; obtaining a set of leakage points of the target container determined in each iteration process; the influence range of the point of highest signal intensity refers to a connected region with all signal intensities greater than a preset signal intensity and centered on the point of highest signal intensity.

2. The method of claim 1, wherein whether the point of highest signal intensity is valid is determined by judging whether the signal-to-noise ratio of the point of highest signal intensity is greater than a preset signal-to-noise ratio threshold and whether the influence range of the point of highest signal intensity is greater than a preset minimum area threshold.

3. The method of claim 2, wherein the signal-to-noise ratio of the point of highest signal intensity is determined based on the following steps: determining the signal intensity of the point of highest signal intensity; determining a noise evaluation region, which is the other region of the acoustic spectrum after shielding the influence range of the point of highest signal intensity; determining the background noise intensity related to the noise evaluation region; determining the signal-to-noise ratio of the point of highest signal intensity based on the signal intensity of the point of highest signal intensity and the background noise intensity.

4. The method of claim 1, wherein The method comprises the following steps: convolving the product image to be tested in horizontal and vertical directions using a first-order differential operator to obtain the gradient amplitude and gradient direction of each pixel point on the product image to be tested; setting the pixel points with a gradient amplitude greater than a preset high amplitude threshold as strong edge points and setting the pixel points with a gradient amplitude less than or equal to the preset high amplitude threshold but greater than a preset low amplitude threshold as weak edge points; based on the gradient direction of each weak edge point, determining part of the weak edge points connected to any strong edge point among all the weak edge points; sequentially connecting the strong edge points and the part of the weak edge points to construct a closed contour of the target container; based on the closed contour, determining the contour region of the target container.

5. The method of claim 1, wherein The method further comprises the following steps: determining a gradient amplitude map of the product image to be tested; performing binaryzation processing on the gradient amplitude map to obtain a binaryzation gradient amplitude map; determining a closed contour of the target container from the binaryzation gradient amplitude map; Based on the closed contour, a contour region of the target container is determined.

6. The method according to claim 4 or 5, wherein The determination of the contour region of the target container based on the closed contour comprises: Based on the contour area, contour level and contour position of each closed contour, a target closed contour is determined. The region where the target closed contour is located is taken as the contour region of the target container.

7. The method of claim 1, wherein The determination of the contour region of the target container in the product image to be tested further comprises: inputting the product image to be tested into a contour labeling model to obtain a contour region probability map output by the contour labeling model; differentially assigning pixel points in the contour region probability map that are greater than or less than a preset probability threshold to obtain a binary mask image; determining the contour region of the target container based on the binary mask image; The contour labeling model is obtained by training based on product image samples, and each product image sample is pre-labeled with a contour region probability map label.

8. The method of claim 1, wherein The extraction of the first sound source signal in the contour region comprises: determining a coordinate system transformation matrix between a microphone sound source coordinate system and a camera image coordinate system; the microphone sound source coordinate system is a spatial coordinate system in which a microphone array for container airtightness detection is located, and the camera image coordinate system is a coordinate system in which a camera for capturing the product image to be tested is located; generating a full-space acoustic map covering the container airtightness detection space based on acoustic signals collected by the microphone array; traversing each potential sound source position point in the full-space acoustic map; projecting microphone sound source coordinates of each potential sound source position point in the microphone sound source coordinate system to the camera image coordinate system using the coordinate system transformation matrix to obtain corresponding image pixel coordinates; determining whether the image pixel coordinates corresponding to each potential sound source position point are located within the contour region of the target container; if the image pixel coordinates are located within the contour region, the region corresponding to the potential sound source position point in the full-space acoustic map is retained; generating the first sound source signal based on all retained regions in the full-space acoustic map.

9. The method of claim 8, wherein The determination of the coordinate system transformation matrix between the microphone sound source coordinate system and the camera image coordinate system comprises: selecting a calibration object provided with a visual calibration pattern and an acoustic beacon, and the physical position relationship of the acoustic beacon relative to the visual calibration pattern is predetermined; capturing the calibration object by the camera and identifying the visual calibration pattern to determine the pose of the visual calibration pattern in the camera image coordinate system; acquiring the acoustic signals emitted by the acoustic beacon through the microphone array to locate the coordinates of the acoustic beacon in the microphone sound source coordinate system; calculating the coordinate system transformation matrix based on the pose of the visual calibration pattern in the camera image coordinate system, the coordinates of the acoustic beacon in the microphone sound source coordinate system, and the physical position relationship.

10. The method of claim 8, wherein The determination of the weld position region in the contour region comprises: obtaining a process design model of the target container; Based on the process design model, a sequence of three-dimensional coordinate points of all welds of the target container is obtained; According to a pose transformation matrix between a model coordinate system of the process design model and a camera image coordinate system, the sequence of three-dimensional coordinate points is projected onto the product image to be tested to obtain weld line segments of all welds of the target container on the product image to be tested; Based on the weld line segments, a weld mask is generated on the product image to be tested; An intersection of the weld mask and the contour region is determined as the weld position region.

11. The method of claim 10, wherein The extracting of the second sound source signal from the first sound source signal in the weld position region comprises: traversing each potential sound source position point in an acoustic spectrum corresponding to the first sound source signal; projecting a microphone sound source coordinate of each potential sound source position point in the microphone sound source coordinate system to the camera image coordinate system by using the coordinate system transformation matrix to obtain a corresponding image pixel coordinate; if the image pixel coordinate is located in the weld position region, retaining a region corresponding to the potential sound source position point in the acoustic spectrum corresponding to the first sound source signal; generating the second sound source signal based on all retained regions in the acoustic spectrum corresponding to the first sound source signal.

12. The method of claim 1, wherein The target container is an inner container of an electric water heater.

13. The method of claim 1, wherein A solvent with anti-rust, strong tension and volatile characteristics is sprayed on the surface of the target container.

14. A container tightness testing device, characterized by It comprises: a container positioning unit configured to determine a contour region of a target container in a product image to be tested; a sound source signal preliminary screening unit configured to extract a first sound source signal in the contour region; a weld positioning unit configured to determine a weld position region in the contour region; a sound source signal re-screening unit configured to extract a second sound source signal in the weld position region from the first sound source signal; a leak point positioning unit configured to locate a possible leak point on the target container based on the second sound source signal, specifically comprising: obtaining an acoustic spectrum corresponding to the second sound source signal; determining a point of highest signal intensity on the acoustic spectrum; in a case where the point of highest signal intensity is determined to be valid, determining a leak point of the target container according to a position of the point of highest signal intensity in the acoustic spectrum; masking an influence range of the point of highest signal intensity in the acoustic spectrum; iteratively performing the determination of the point of highest signal intensity on the acoustic spectrum and the masking of the influence range of the point of highest signal intensity in the acoustic spectrum until the point of highest signal intensity is determined to be invalid; obtaining a set of leak points of the target container determined in each iteration process; the influence range of the point of highest signal intensity refers to a connected region with all signal intensities greater than a preset signal intensity centered on the point of highest signal intensity.

15. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, The processor executes the computer program to implement the container air tightness detection method of any one of claims 1 to 13. 16.A non-transitory computer-readable storage medium having stored thereon a computer program. The computer program is executed by the processor to implement the container air tightness detection method of any one of claims 1 to 13.

17. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method for detecting the air tightness of the container according to any one of claims 1 to 13.

Images