A method, system and readable storage medium for monitoring a welding camera and radar

Abstract

The application discloses a kind of welding camera and radar monitoring method, system and readable storage medium, belong to image recognition technical field, this method includes: in the case where determining that the device to be welded is in the welding stage, the image group of the device to be welded is obtained;According to image group, the sharpness change rate is obtained using image analysis algorithm;The sharpness change rate is input into the sharpness stress mapping model to obtain the stress change of the device to be welded.The method does not require a large amount of computing resources, and is not limited by the complexity and accuracy of the simulation model, can accurately and efficiently predict the stress change in the welding process.

Description

A monitoring method, system, and readable storage medium for welding cameras and radar. Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a monitoring method, system, and readable storage medium for welding cameras and radar. Background Technology

[0002] With the deepening development of automotive electronics and intelligence, in-vehicle cameras, as an important sensor, have significantly impacted vehicle performance and safety. A common method in the production of these cameras is based on the principle of symmetry, welding multiple points simultaneously (although welding multiple points at the same time is not strictly simultaneous, but rather relative; that is, these welding points are completed sequentially within a very short time range, such as 0.1, 0.3, and 0.5 seconds). This method can significantly improve production efficiency, but it also leads to a problem: changes in material stress during the welding process.

[0003] Currently, in order to monitor stress changes during the welding process, computer simulation methods in related technologies need to predict stress changes by simulating the welding process. However, these computer simulation methods require a large amount of computational resources, and their accuracy is limited by the complexity and accuracy of the simulation model. Summary of the Invention

[0004] This application provides a monitoring method, system, and readable storage medium for welding cameras and radar. This method does not require a large amount of computing resources and is not limited by the complexity and accuracy of simulation models. It can accurately and efficiently predict stress changes during the welding process.

[0005] In a first aspect, this application provides a monitoring method for welding cameras and radar, the method comprising: acquiring a group of images of the device to be welded when it is determined that the device to be welded is in the welding stage; obtaining a sharpness change rate based on the image group using an image resolution algorithm; and inputting the sharpness change rate into a sharpness stress mapping model to obtain the stress change of the device to be welded.

[0006] In the above embodiments, by acquiring a group of images of the device to be welded, using an image resolution algorithm to obtain the sharpness change rate, and inputting the sharpness change rate into a sharpness stress mapping model, the stress change of the device to be welded is obtained. This method can observe and monitor stress changes during the welding process in real time and intuitively. Compared with existing computer simulation methods, this method does not require a large amount of computing resources and is not limited by the complexity and accuracy of the simulation model. It can accurately and efficiently predict stress changes during the welding process, thereby improving welding quality and optimizing production efficiency.

[0007] In conjunction with some embodiments of the first aspect, in some embodiments, the image group specifically includes: a first image containing the plane to be inspected of the device to be welded before welding the device to be welded; and a second image containing the plane to be inspected after welding the device to be welded.

[0008] In the above embodiments, by comparing images before and after welding, the stress changes of the welding device can be observed and analyzed intuitively. At the same time, by only acquiring images before and after welding, the influence of the irradiation intensity changes caused by sparks during the welding process on the image clarity can be effectively avoided.

[0009] In conjunction with some embodiments of the first aspect, in some embodiments, the image group further includes: when welding the device to be welded, taking a number of third images of the plane to be inspected at preset time intervals.

[0010] In the above embodiments, by taking several third images of the plane to be inspected at preset time intervals during the welding process, more information about the welding process can be obtained. This information can help to clearly understand at which stage of the welding process the stress changes, thereby providing the possibility of a deeper understanding and optimization of the welding process.

[0011] In conjunction with some embodiments of the first aspect, in some embodiments, the image group further includes: a fourth image surrounding the welding device corresponding to the time the first image was captured; a fifth image surrounding the welding device corresponding to the time the second image was captured; and several sixth images surrounding the welding device corresponding one-to-one to the times the several third images were captured. Obtaining the sharpness change rate using an image resolution algorithm based on the image group specifically includes: determining a first sharpness change rate of the plane to be detected based on the first image, several third images, and the second image; determining a second sharpness change rate around the welding device based on the fourth image, several sixth images, and the fifth image; and determining a sharpness change rate based on the first and second sharpness change rates.

[0012] In the above embodiments, the image set also includes images related to the environment surrounding the welding apparatus, thus taking environmental factors, including changes in Martian illumination intensity, into account when calculating the rate of change in image sharpness. By comparing the rate of change in sharpness of the plane under test and the area surrounding the welding apparatus, the sharpness changes caused by stress variations can be analyzed more accurately, thereby eliminating the influence of Martian illumination intensity variations on image sharpness and improving the accuracy of stress change prediction during the welding process.

[0013] In conjunction with some embodiments of the first aspect, in some embodiments, the image group further includes: a first predicted image corresponding to the first image capture time, the first predicted image being mapped from all images surrounding the welding device corresponding to the first image capture time; a second predicted image corresponding to the second image capture time, the second predicted image being mapped from all images surrounding the welding device corresponding to the second image capture time; and several third predicted images corresponding one-to-one with several third image capture times, the third predicted images being mapped from all images surrounding the welding device corresponding to the third image capture times. Obtaining the sharpness change rate using an image resolution algorithm based on the image group specifically includes: determining a first sharpness change rate of the plane to be detected based on the first image, several third images, and the second image; determining a third sharpness change rate of the plane to be detected based on the first predicted image, several third predicted images, and the second predicted image; and determining a sharpness change rate based on the first sharpness change rate and the third sharpness change rate.

[0014] In the above embodiments, the predicted image is obtained by mapping all images around the welding device. It can simulate the environmental state at the same point in time as the plane to be inspected, without the influence of the device itself. In this way, the rate of image sharpness change caused by environmental changes can be calculated more accurately. Thus, the rate of sharpness change of the original image and the predicted image can be compared, thereby more accurately analyzing the rate of image sharpness change caused by the device itself.

[0015] In conjunction with some embodiments of the first aspect, in some embodiments, before determining the first sharpening change rate of the plane to be detected based on the first predicted image, a plurality of third predicted images, and the second predicted image, the method further includes: registering the first predicted image with the first image; registering the second predicted image with the second image; and registering the plurality of third predicted images with a plurality of third images corresponding to each other.

[0016] In the above embodiments, by registering the predicted image with the actual image before determining the sharpness change rate of the plane to be detected, the consistency of the spatial layout between the predicted and actual images can be ensured. This registration step can correct for possible regional differences between the predicted and actual images, further improving the accuracy of the sharpness change rate calculation. This method compensates for the problem that the predicted image may not match the actual image region.

[0017] In conjunction with some embodiments of the first aspect, in some embodiments, before acquiring an image group of the device to be welded when it is determined that the device to be welded is in the welding stage, the method further includes: determining all pixels of the first image as unfilled pixels; selecting one unfilled pixel of the first image; determining a texture block in all images around the welding device corresponding to the time the first image was captured that is most similar to the pixels surrounding the unfilled pixel; assigning the average pixel value of the texture block to the unfilled pixel; determining the unfilled pixel as a filled pixel; and skipping to the step of selecting one unfilled pixel of the first image until all pixels of the first image are determined to be filled pixels.

[0018] In the above embodiments, before acquiring the image group of the device to be welded, the image prediction operation is completed by filling the pixels of the first image with the average pixel value of the texture block.

[0019] Secondly, this application provides a monitoring system for welding cameras and radar, the monitoring system including a server, the server comprising:

[0020] The acquisition module is used to acquire a group of images of the device to be welded when it is determined that the device is in the welding stage.

[0021] The sharpening module is used to obtain the sharpening rate of change based on the image group using an image resolution algorithm;

[0022] The stress module is used to input the sharpening change rate into the sharpness stress mapping model to obtain the stress change of the device to be welded.

[0023] In conjunction with some embodiments of the second aspect, in some embodiments, the image group specifically includes:

[0024] A first image containing the plane to be inspected of the device to be welded, prior to welding the device to be welded;

[0025] After welding the device to be welded, a second image containing the plane to be inspected is generated.

[0026] In conjunction with some embodiments of the second aspect, in some embodiments, the image group further includes:

[0027] When welding the device to be welded, several third images of the plane to be inspected are taken sequentially at preset time intervals.

[0028] In conjunction with some embodiments of the second aspect, in some embodiments, the image group further includes:

[0029] A fourth image of the welding device corresponding to the time the first image was captured;

[0030] A fifth image of the welding device corresponding to the time the second image was captured;

[0031] Several sixth images surrounding the welding device, corresponding one-to-one with the shooting time of several third images;

[0032] The de-sharpening module specifically includes:

[0033] The first sharpening submodule is used to determine the first sharpening change rate of the plane to be detected based on the first image, several third images, and the second image;

[0034] The second sharpening submodule is used to determine the second sharpening change rate around the welding device based on the fourth image, several sixth images, and the fifth image;

[0035] The third sharpening submodule is used to determine the sharpening change rate based on the first sharpening change rate and the second sharpening change rate.

[0036] In conjunction with some embodiments of the second aspect, in some embodiments, the image group further includes:

[0037] A first predicted image corresponding to the first image capture time, the first predicted image being obtained by mapping all images around the welding device corresponding to the first image capture time;

[0038] A second predicted image corresponding to the second image capture time, the second predicted image being obtained by mapping all images around the welding device corresponding to the second image capture time;

[0039] Several third predicted images correspond one-to-one with several third image capture times. The third predicted images are obtained by mapping all images around the welding device corresponding to the third image capture time.

[0040] The de-sharpening module specifically includes:

[0041] The fourth sharpening submodule is used to determine the first sharpening change rate of the plane to be detected based on the first image, several third images, and the second image;

[0042] The fifth sharpening submodule is used to determine the third sharpening change rate of the plane to be detected based on the first predicted image, several third predicted images, and the second predicted image.

[0043] The sixth sharpening submodule is used to determine the sharpening change rate based on the first sharpening change rate and the third sharpening change rate.

[0044] In conjunction with some embodiments of the second aspect, in some embodiments, the server further includes:

[0045] The first registration module is used to register the first predicted image with the first image;

[0046] The second registration module is used to register the second predicted image with the second image;

[0047] The third registration module is used to register several third predicted images with several corresponding third images.

[0048] In conjunction with some embodiments of the second aspect, in some embodiments, the server further includes:

[0049] The first determining module is used to determine all pixels of the first image as unfilled pixels;

[0050] The selection module is used to select an unfilled pixel in the first image;

[0051] The second determining module is used to determine a texture block that is most similar to the pixels surrounding the unfilled pixel in all images around the welding device corresponding to the first image capture time.

[0052] The assignment module is used to assign the average pixel value of the texture block to the unfilled pixel.

[0053] The third determining module is used to determine the unfilled pixel as a filled pixel.

[0054] The jump module is used to jump to the step of selecting an unfilled pixel in the first image until all pixels in the first image are determined to be filled pixels.

[0055] Thirdly, embodiments of this application provide a monitoring system for welding cameras and radar, the system comprising: one or more processors and a memory;

[0056] The memory is coupled to the one or more processors and is used to store computer program code, which includes computer instructions that the one or more processors call to cause the monitoring system of the welding camera and radar to perform the methods described in the first aspect and any possible implementation thereof.

[0057] Fourthly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a server, cause the server to perform the method described in the first aspect and any possible implementation thereof.

[0058] Fifthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a server, cause the server to perform the method described in the first aspect and any possible implementation thereof.

[0059] Understandably, the welding camera and radar monitoring system provided in the second aspect, the third aspect, the fourth aspect, and the fifth aspect are all used to execute the welding camera and radar monitoring method provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0060] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0061] 1. The welding camera and radar monitoring method provided in this application acquires a group of images of the device to be welded, obtains the sharpness change rate using an image resolution algorithm, and inputs the sharpness change rate into a sharpness stress mapping model to obtain the stress change of the device to be welded. This method can observe and monitor stress changes during the welding process in real time and intuitively. Compared with existing computer simulation methods, this method does not require a large amount of computing resources and is not limited by the complexity and accuracy of the simulation model. It can accurately and efficiently predict stress changes during the welding process, thereby improving welding quality and optimizing production efficiency.

[0062] 2. The welding camera and radar monitoring method provided in this application can intuitively observe and analyze the stress changes of the welding device by comparing images before and after welding. At the same time, by acquiring only images before and after welding, the influence of the irradiation intensity changes caused by sparks during welding on the image clarity is effectively avoided.

[0063] 3. The welding camera and radar monitoring method provided in this application can obtain more information about the welding process by sequentially taking several third images of the plane to be inspected at preset time intervals during the welding process. This information can help to clearly understand at which stage of the welding process the stress changes, thereby providing the possibility of a deeper understanding and optimization of the welding process.

[0064] 4. The monitoring method for welding cameras and radar provided in this application includes an image set that also contains images related to the environment surrounding the welding device. This allows environmental factors, including changes in Martian illumination intensity, to be considered when calculating the rate of change in image sharpness. By comparing the rate of change in sharpness between the plane to be inspected and the area surrounding the welding device, the change in sharpness caused by stress variations can be analyzed more accurately, thereby eliminating the influence of Martian illumination intensity variations on image sharpness and improving the accuracy of stress change prediction during the welding process.

[0065] 5. The monitoring method for welding cameras and radar provided in this application predicts an image by mapping all images around the welding device. This simulates the environmental state at the same point in time as the plane to be inspected, without the influence of the device itself. In this way, the rate of image sharpness change caused by environmental changes can be calculated more accurately. Thus, the rate of sharpness change between the original image and the predicted image can be compared, allowing for a more accurate analysis of the rate of image sharpness change caused by the device itself. Attached Figure Description

[0066] Figure 1 is a flowchart illustrating the monitoring method for welding cameras and radar provided in this application.

[0067] Figure 2 is another flowchart illustrating the monitoring method for welding cameras and radar provided in this application.

[0068] Figure 3 is another flowchart illustrating the monitoring method for welding cameras and radar provided in this application.

[0069] Figure 4 is another flowchart illustrating the monitoring method for welding cameras and radar provided in this application.

[0070] Figure 5 is another flowchart illustrating the monitoring method for welding cameras and radar provided in this application.

[0071] Figure 6 is a schematic diagram of the modular virtual device of the welding camera and radar monitoring system provided in this application.

[0072] Figure 7 is a schematic diagram of the physical device of the monitoring system of welding camera and radar provided in this application. Detailed Implementation

[0073] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0074] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0075] The monitoring method using a welding camera and radar in this embodiment is described below:

[0076] As shown in Figure 1, Figure 1 is a flowchart illustrating the monitoring method of the welding camera and radar provided in this application.

[0077] S101. If it is determined that the device to be welded is in the welding stage, acquire a group of images of the device to be welded.

[0078] It is obvious that it is necessary to confirm that the device to be welded is already in the welding stage. This is because during the welding stage, the device will experience specific stress changes, causing slight deformation of the device to be welded.

[0079] The image set can be acquired via a camera. When acquiring the image set, it is important to ensure that the image quality is sufficiently clear. Preferably, the image should be focused on the plane of the device to be welded, as the plane of the device is highly sensitive to stress changes, which will directly affect the accuracy of subsequent image resolution algorithms.

[0080] Although welding multiple points occurs at the same time, this is not simultaneity in the strict sense, but rather relative simultaneity. That is to say, these welding points will be completed sequentially within a very short time range, such as 0.1, 0.3, and 0.5 seconds.

[0081] S102. Obtain the sharpening rate of change based on the image group using an image resolution algorithm.

[0082] In this embodiment, the image resolution algorithm is SFR, or Spatial Frequency Response, and its specific steps are as follows: 0. Obtain the ROI with the vertical hypotenuse; 1. Normalize the data; 2. Calculate the pixel centroid of each row of the image; 3. Perform linear fitting on the centroid of each row using the least squares method to obtain a straight line about the centroid; 4. Relocate the ROI to obtain ESF; 5. Perform four-fold oversampling on the obtained ESF; 6. Obtain LSF through difference operations; 7. Apply a Hamming window to the LSF; 8. Perform DFT operations. The specific use and principle of the image resolution algorithm are relatively mature and will not be limited here.

[0083] Continuing the previous example, when a camera focuses, it adjusts its focus based on the object's distance, shape, and size. If the object deforms, it may change the distance between the object and the camera, or the object's shape and size, which could lead to inaccurate focusing and affect image sharpness. Simultaneously, the object's shape and surface texture affect the scattering and reflection of light. If the object deforms, it may alter the pattern of light scattering and reflection, potentially changing the image's sharpness.

[0084] S103. Input the sharpness change rate into the sharpness stress mapping model to obtain the stress change of the device to be welded.

[0085] Using the sharpness change rate as input data and stress change as output data, a pre-defined sharpness-stress mapping model is trained. This sharpness-stress mapping model can be any known model, such as a deep neural network, a convolutional neural network, or a recurrent neural network. Preferably, it is a mathematical model, and more specifically, it is a mathematical model based on mapping relationships.

[0086] As can be seen, by acquiring a set of images of the device to be welded, using an image resolution algorithm to obtain the sharpness change rate, and inputting the sharpness change rate into a sharpness stress mapping model, the stress change of the device to be welded can be obtained. This method can observe and monitor stress changes during the welding process in real time and intuitively. Compared with existing computer simulation methods, this method does not require a large amount of computational resources and is not limited by the complexity and accuracy of the simulation model. It can accurately and efficiently predict stress changes during the welding process, thereby improving welding quality and optimizing production efficiency.

[0087] The embodiments of this application will be described in more detail below using four more specific examples, in conjunction with the embodiments shown in Figures 2 to 5:

[0088] The first embodiment is shown in Figure 2, which is a flowchart illustrating a monitoring method for welding cameras and radar provided in this application.

[0089] S201. When it is determined that the device to be welded is in the welding stage, an image set of the device to be welded is acquired. The image set includes: a first image containing the plane to be inspected of the device to be welded before welding; and a second image containing the plane to be inspected after welding the device to be welded.

[0090] It's important to note that the welding process generates numerous sparks, which emit intense light. Variations in this light intensity can affect image sharpness. For example, when sparks illuminate the surface being inspected, they can cause overexposure, making certain parts of the image too bright and reducing sharpness. Conversely, when the sparks dim, the image may become too dark, increasing noise and also reducing sharpness.

[0091] Therefore, it can be inferred that the first image and the second image are images before and after welding, respectively, and thus the first image and the second image do not contain noise generated by sparks.

[0092] S202. Obtain the sharpening rate of change based on the first image and the second image using an image resolution algorithm.

[0093] The steps used in this embodiment are based on the same concept as those used in the above embodiments. The specific implementation process is detailed in step S102, and will not be repeated here.

[0094] S203. Input the sharpness change rate into the sharpness stress mapping model to obtain the stress change of the device to be welded.

[0095] The steps used in this embodiment are based on the same concept as those used in the above embodiments. The specific implementation process is detailed in step S103, and will not be repeated here.

[0096] As can be seen, by comparing images before and after welding, the stress changes of the welding device can be observed and analyzed intuitively. At the same time, by only acquiring images before and after welding, the influence of changes in irradiation intensity caused by sparks during welding on image clarity can be effectively avoided.

[0097] In practical use, the above embodiments present a new problem: while pre- and post-weld images can indeed help understand the overall stress changes, the lack of images during the welding process makes it impossible to determine at which stage of the welding process these stress changes occur. This may limit a deeper understanding and optimization of the welding process. For example, it may be impossible to determine which stage experiences the greatest stress change. Therefore, this could lead to missing important opportunities to improve the welding process. Furthermore, if a problem occurs at a certain stage of the welding process, it may not be detected and resolved in a timely manner. Therefore, taking one approach to solving the above problems as an example, and referring to the embodiment shown in Figure 3, the embodiments of this application will be described in more detail:

[0098] The second embodiment is shown in Figure 3, which is a flowchart illustrating the monitoring method of the welding camera and radar provided in this application.

[0099] S301. When it is determined that the device to be welded is in the welding stage, an image group of the device to be welded is acquired. The image group includes: a first image containing the plane to be inspected of the device to be welded before welding; a second image containing the plane to be inspected after welding; and several third images of the plane to be inspected obtained by taking pictures of the plane to be inspected in sequence at preset time intervals when welding the device to be welded.

[0100] The steps used in this embodiment are based on the same concept as those used in the above embodiments. The specific implementation process is detailed in step S201, and will not be repeated here.

[0101] S302. Based on the first image, the second image, and several third images, the sharpening rate is obtained using an image resolution algorithm.

[0102] The first image before welding is input into the image resolution algorithm to obtain the first sharpness. This sharpness can be considered as the initial state of the welding process and will serve as the benchmark for subsequent sharpness change rate calculations. The second image after welding is input into the same image resolution algorithm to obtain the second sharpness. By comparing the first sharpness with the second sharpness, the total sharpness change rate can be obtained, which reflects the impact of the welding process on the device state. Several third images taken sequentially at preset time intervals during the welding process are then input into the image resolution algorithm. For each third image, its sharpness is calculated and compared with the sharpness of the previous image to obtain the sharpness change rate over that time period.

[0103] Therefore, not only can the clarity change rate before and after welding be obtained, but also the clarity change rate for each preset time period during the welding process. This clarity change rate data can provide detailed information on stress changes during the welding process, helping to better understand and control the welding process.

[0104] S303. Input the sharpness change rate into the sharpness stress mapping model to obtain the stress change of the device to be welded.

[0105] The steps used in this embodiment are based on the same concept as those used in the above embodiments. The specific implementation process is detailed in step S103, and will not be repeated here.

[0106] It is evident that by taking several third images of the plane to be inspected at preset time intervals during the welding process, more information about the welding process can be obtained. This information can help to clearly understand at which stage of the welding process the stress changes, thus providing the possibility of a deeper understanding and optimization of the welding process.

[0107] The above embodiments provide the possibility of a deeper understanding and optimization of the welding process. However, in actual use, a large number of sparks are generated during welding, which emit intense light. This variation in light intensity can affect image clarity. For example, when sparks illuminate the surface to be inspected, it may cause image overexposure, making certain parts of the image too bright, thus reducing image clarity. Conversely, when the sparks disappear, the image may become too dark, increasing image noise and also reducing image clarity. Therefore, taking one method to solve the above problems as an example, and referring to the embodiment shown in Figure 4, the embodiments of this application will be described in more detail:

[0108] The third embodiment is shown in Figure 4, which is a flowchart illustrating the monitoring method of the welding camera and radar provided in this application.

[0109] S401. When it is determined that the device to be welded is in the welding stage, an image group of the device to be welded is acquired. The image group includes: a first image containing the plane to be inspected of the device to be welded before welding; a second image containing the plane to be inspected after welding; several third images of the plane to be inspected obtained by taking pictures of the device to be inspected in sequence at preset time intervals during welding; a fourth image of the area around the welding device corresponding to the time of taking the first image; a fifth image of the area around the welding device corresponding to the time of taking the second image; and several sixth images of the area around the welding device corresponding one-to-one to the time of taking the several third images.

[0110] The first image will serve as a baseline image for reference and comparison, while the fourth image will be used to record the ambient lighting conditions at the start of the welding process.

[0111] Similarly, the second and fifth images will be used to compare with the baseline image to evaluate the impact of the welding process on the surface to be inspected and ambient lighting conditions.

[0112] Similarly, the plane to be inspected is photographed sequentially at preset time intervals to obtain several third images, as well as several sixth images around the welding device that correspond one-to-one with the shooting time of these third images. These images will be used to monitor dynamic changes during the welding process.

[0113] S402. Determine the first sharpening change rate of the plane to be detected based on the first image, several third images, and the second image.

[0114] The steps used in this embodiment are based on the same concept as those used in the above embodiments. The specific implementation process is detailed in step S302, and will not be repeated here.

[0115] S403. Determine the second clarity change rate around the welding device based on the fourth image, several sixth images, and the fifth image.

[0116] The steps used in this embodiment are based on the same concept as those used in the above embodiments. The specific implementation process is detailed in step S302, and will not be repeated here.

[0117] S404. Determine the sharpening rate based on the first sharpening rate and the second sharpening rate.

[0118] This process involves comparing and integrating a first and a second rate of sharpness change to attempt to eliminate the influence of lighting variations and more accurately reflect the true changes in the plane being tested. Specifically, the first and second rates of sharpness change are first compared, then the first rate of sharpness change is adjusted based on the comparison results to eliminate the influence of lighting variations. Finally, the adjusted first rate of sharpness change is taken as the final rate of sharpness change.

[0119] S405. Input the sharpness change rate into the sharpness stress mapping model to obtain the stress change of the device to be welded.

[0120] The steps used in this embodiment are based on the same concept as those used in the above embodiments. The specific implementation process is detailed in step S103, and will not be repeated here.

[0121] As can be seen, the image set also includes images related to the environment surrounding the welding apparatus. This allows environmental factors, including variations in Martian illumination intensity, to be considered when calculating the rate of change in image sharpness. By comparing the rate of change in sharpness between the plane under test and the area surrounding the welding apparatus, the changes in sharpness caused by stress variations can be analyzed more accurately. This eliminates the impact of variations in Martian illumination intensity on image sharpness and improves the accuracy of stress change prediction during the welding process.

[0122] The above embodiments eliminate the impact of variations in Martian illumination intensity on image clarity. However, in actual use, since the area around the plane to be inspected and the welding device are not the same, certain errors may still exist. Therefore, taking one method to solve the above problem as an example, and referring to the embodiment shown in Figure 5, the embodiments of this application will be described in more detail:

[0123] The fourth embodiment is shown in Figure 5, which is a flowchart illustrating the monitoring method of the welding camera and radar provided in this application.

[0124] S501. When it is determined that the device to be welded is in the welding stage, an image group of the device to be welded is acquired. The image group includes: a first image containing the plane to be inspected of the device to be welded before welding; a second image containing the plane to be inspected after welding; several third images obtained by sequentially taking pictures of the plane to be inspected at preset time intervals when welding the device to be welded; a first predicted image corresponding to the time of the first image taking, the first predicted image being mapped from all images around the welding device corresponding to the time of the first image taking; a second predicted image corresponding to the time of the second image taking, the second predicted image being mapped from all images around the welding device corresponding to the time of the second image taking; and several third predicted images corresponding one-to-one with the times of several third images taking, the third predicted images being mapped from all images around the welding device corresponding to the times of the third images taking.

[0125] The specific method for obtaining the first predicted image corresponding to the first image capture time, which is obtained by mapping all images around the welding device corresponding to the first image capture time, is as follows:

[0126] All pixels in the first image are identified as unfilled pixels; one unfilled pixel in the first image is selected; a texture block most similar to the pixels surrounding the unfilled pixel is identified in all images around the welding device corresponding to the first image capture time; the average pixel value of the texture block is assigned to the unfilled pixel; the unfilled pixel is identified as a filled pixel; the process of selecting one unfilled pixel in the first image is repeated until all pixels in the first image are identified as filled pixels.

[0127] As can be seen, before acquiring the image group of the device to be welded, the image prediction operation is completed by filling the pixels of the first image with the average pixel value of the texture block.

[0128] Similarly, the second and third predicted images are obtained in the same way.

[0129] In actual use, the first predicted image, the second predicted image, and the third predicted image may have regional differences from the first image, the second image, and the third image. Therefore, based on the above technical issues, the first predicted image is registered with the first image; the second predicted image is registered with the second image; and several third predicted images are registered with several corresponding third images.

[0130] Image registration is a common computer vision task used to align two or more images so that they correspond to each other in space.

[0131] A specific implementation involves finding corner points and edge regions in each image, then calculating a geometric transformation to align one image to another. This geometric transformation could be an affine transformation, a projective transformation, or another type of transformation.

[0132] As can be seen, by registering the predicted image with the actual image before determining the sharpness change rate of the plane to be detected, the consistency of the spatial layout between the predicted and actual images can be ensured. This registration step can correct for potential regional differences between the predicted and actual images, further improving the accuracy of the sharpness change rate calculation. This method compensates for the problem of inconsistencies between the predicted and actual image regions.

[0133] S502. Determine the first sharpening change rate of the plane to be detected based on the first image, several third images, and the second image.

[0134] The steps used in this embodiment are based on the same concept as those used in the above embodiments. The specific implementation process is detailed in step S302, and will not be repeated here.

[0135] S503. Determine the third sharpening rate of the plane to be detected based on the first predicted image, several third predicted images, and the second predicted image.

[0136] The steps used in this embodiment are based on the same concept as those used in the above embodiments. The specific implementation process is detailed in step S302, and will not be repeated here.

[0137] S504. Determine the sharpening rate based on the first sharpening rate and the third sharpening rate.

[0138] The steps used in this embodiment are based on the same concept as those used in the above embodiments. The specific implementation process is detailed in step S404, and will not be repeated here.

[0139] S505. Input the sharpness change rate into the sharpness stress mapping model to obtain the stress change of the device to be welded.

[0140] The steps used in this embodiment are based on the same concept as those used in the above embodiments. The specific implementation process is detailed in step S103, and will not be repeated here.

[0141] As can be seen, the predicted image is obtained by mapping all images around the welding device. It can simulate the environmental state at the same point in time as the plane to be inspected, without the influence of the device itself. In this way, the rate of image sharpness change caused by environmental changes can be calculated more accurately. Thus, the rate of sharpness change between the original image and the predicted image can be compared, thereby more accurately analyzing the rate of image sharpness change caused by the device itself.

[0142] The following are device embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the device embodiments of this application, please refer to the method embodiments of this application.

[0143] Referring to Figure 6, this application embodiment provides a monitoring system for welding cameras and radar. The monitoring system for welding cameras and radar includes a server, which includes:

[0144] The servers include:

[0145] The acquisition module 601 is used to acquire a group of images of the device to be welded when it is determined that the device to be welded is in the welding stage.

[0146] The sharpening module 602 is used to obtain the sharpening change rate based on the image group using an image resolution algorithm;

[0147] Stress module 603 is used to input the sharpening change rate into the sharpness stress mapping model to obtain the stress change of the device to be welded.

[0148] In some embodiments, the image group specifically includes:

[0149] A first image containing the plane to be inspected of the device to be welded, prior to welding the device to be welded;

[0150] After welding the device to be welded, a second image containing the plane to be inspected is generated.

[0151] In some embodiments, the image group further includes:

[0152] When welding the device to be welded, several third images of the plane to be inspected are taken sequentially at preset time intervals.

[0153] In some embodiments, the image group further includes:

[0154] A fourth image of the welding device corresponding to the time the first image was captured;

[0155] A fifth image of the welding device corresponding to the time the second image was captured;

[0156] Several sixth images surrounding the welding device, corresponding one-to-one with the shooting time of several third images;

[0157] The de-sharpening module specifically includes:

[0158] The first sharpening submodule is used to determine the first sharpening change rate of the plane to be detected based on the first image, several third images, and the second image;

[0159] The second sharpening submodule is used to determine the second sharpening change rate around the welding device based on the fourth image, several sixth images, and the fifth image;

[0160] The third sharpening submodule is used to determine the sharpening change rate based on the first sharpening change rate and the second sharpening change rate.

[0161] In some embodiments, the image group further includes:

[0162] A first predicted image corresponding to the first image capture time, the first predicted image being obtained by mapping all images around the welding device corresponding to the first image capture time;

[0163] A second predicted image corresponding to the second image capture time, the second predicted image being obtained by mapping all images around the welding device corresponding to the second image capture time;

[0164] Several third predicted images correspond one-to-one with several third image capture times. The third predicted images are obtained by mapping all images around the welding device corresponding to the third image capture time.

[0165] The de-sharpening module specifically includes:

[0166] The fourth sharpening submodule is used to determine the first sharpening change rate of the plane to be detected based on the first image, several third images, and the second image;

[0167] The fifth sharpening submodule is used to determine the third sharpening change rate of the plane to be detected based on the first predicted image, several third predicted images, and the second predicted image.

[0168] The sixth sharpening submodule is used to determine the sharpening change rate based on the first sharpening change rate and the third sharpening change rate.

[0169] In some embodiments, the server further includes:

[0170] The first registration module is used to register the first predicted image with the first image;

[0171] The second registration module is used to register the second predicted image with the second image;

[0172] The third registration module is used to register several third predicted images with several corresponding third images.

[0173] In some embodiments, the server further includes:

[0174] The first determining module is used to determine all pixels of the first image as unfilled pixels;

[0175] The selection module is used to select an unfilled pixel in the first image;

[0176] The second determining module is used to determine a texture block that is most similar to the pixels surrounding the unfilled pixel in all images around the welding device corresponding to the first image capture time.

[0177] The assignment module is used to assign the average pixel value of the texture block to the unfilled pixel.

[0178] The third determining module is used to determine the unfilled pixel as a filled pixel.

[0179] The jump module is used to jump to the step of selecting an unfilled pixel in the first image until all pixels in the first image are determined to be filled pixels.

[0180] This application also discloses a monitoring system for welding cameras and radar. Referring to FIG7, it is a schematic diagram of the physical device of the monitoring system for welding cameras and radar provided in this application. The server 700 may include: at least one processor 701, at least one network interface 704, a user interface 703, a memory 705, and at least one communication bus 702.

[0181] The communication bus 702 is used to enable communication between these components.

[0182] The user interface 703 may include a display screen and a camera. Optionally, the user interface 703 may also include a standard wired interface and a wireless interface.

[0183] The network interface 704 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0184] The processor 701 may include one or more processing cores. The processor 701 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 705, and by calling data stored in memory 705. Optionally, the processor 701 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 701 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 701 and may be implemented as a separate chip.

[0185] The memory 705 may include random access memory (RAM) or read-only memory. Optionally, the memory 705 may include a non-transitory computer-readable storage medium. The memory 705 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 705 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 705 may also be at least one storage device located remotely from the aforementioned processor 701. Referring to FIG5, the memory 705, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and applications for monitoring welding cameras and radar.

[0186] In the server 700 shown in Figure 5, the user interface 703 is mainly used to provide an input interface for the user and obtain user input data; while the processor 701 can be used to call the welding camera and radar monitoring application stored in the memory 705. When executed by one or more processors 701, the server 700 performs one or more of the methods described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0187] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0188] In the various embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0189] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0190] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0191] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0192] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the specification and the disclosure of practical truths.

[0193] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A monitoring method for welding a camera and radar, characterized in that, The method includes: acquiring an image set of the device to be welded when it is determined that the device is in the welding stage; before welding the device, acquiring a first image containing a plane to be inspected of the device; after welding the device, acquiring a second image containing the plane to be inspected; during welding the device, sequentially capturing a plurality of third images of the plane to be inspected at preset time intervals; a first predicted image corresponding to the time of capturing the first image, the first predicted image being mapped from all images around the welding device corresponding to the time of capturing the first image; a second predicted image corresponding to the time of capturing the second image, the second predicted image being mapped from all images around the welding device corresponding to the time of capturing the second image; and a plurality of third images. A plurality of third predicted images are generated, each corresponding to a specific time of the third image capture. These third predicted images are obtained by mapping all images surrounding the welding device at the time the third image was captured. A sharpness change rate is obtained using an image resolution algorithm based on the image group. Specifically: a first sharpness change rate of the plane to be detected is determined based on the first image, the plurality of third images, and the second image; a third sharpness change rate of the plane to be detected is determined based on the first predicted image, the plurality of third predicted images, and the second predicted image; the sharpness change rate is determined based on the first and third sharpness change rates; and the sharpness change rate is input into a sharpness stress mapping model to obtain the stress change of the welding device.

2. The monitoring method using a welding camera and radar according to claim 1, characterized in that, The image set further includes: a fourth image surrounding the welding device corresponding to the shooting time of the first image; a fifth image surrounding the welding device corresponding to the shooting time of the second image; and several sixth images surrounding the welding device corresponding one-to-one with the shooting times of the several third images. The step of obtaining the sharpness change rate using an image resolution algorithm based on the image set specifically includes: determining a first sharpness change rate of the plane to be detected based on the first image, the several third images, and the second image; determining a second sharpness change rate around the welding device based on the fourth image, the several sixth images, and the fifth image; and determining the sharpness change rate based on the first sharpness change rate and the second sharpness change rate.

3. The monitoring method using a welding camera and radar according to claim 1, characterized in that, Before determining the first sharpening change rate of the plane to be detected based on the first predicted image, a plurality of the third predicted images, and the second predicted image, the method further includes: registering the first predicted image with the first image; registering the second predicted image with the second image; and registering the plurality of the third predicted images with the plurality of third images that correspond one-to-one with each other.

4. The monitoring method using a welding camera and radar according to claim 1, characterized in that, Before acquiring an image set of the device to be welded when it is determined that the device is in the welding stage, the method further includes: determining all pixels of the first image as unfilled pixels; selecting one unfilled pixel of the first image; determining a texture block most similar to the pixels surrounding the unfilled pixel in all images around the welding device corresponding to the time the first image was captured; assigning the average pixel value of the texture block to the unfilled pixel; and determining the unfilled pixel as a filled pixel. Skip to the step of selecting an unfilled pixel in the first image until all pixels in the first image are determined to be filled pixels.

5. A monitoring system for welding cameras and radar, the monitoring system comprising a server, characterized in that, The server, applied to any one of claims 1-4, comprises: an acquisition module for acquiring an image set of the device to be welded when it is determined that the device to be welded is in the welding stage; a sharpening module for obtaining a sharpening change rate based on the image set using an image resolution algorithm; and a stress module for inputting the sharpening change rate into a sharpness stress mapping model to obtain the stress change of the device to be welded.

6. A monitoring system for welding cameras and radar, characterized in that, include: One or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the monitoring system of the welding camera and radar to perform the method as described in any one of claims 1-4.

7. A computer-readable storage medium comprising instructions, characterized in that, When the instruction is executed on the monitoring system of the welding camera and radar, the monitoring system of the welding camera and radar performs the method as described in any one of claims 1-4.

Images