A real-time monitoring method for engine production line assembly

Abstract

The present application relates to the technical field of image processing, and more particularly to a real-time monitoring method for engine production line assembly, which can capture subtle changes in the assembly process, discover problems such as missing and misassembly in time, significantly reduce the risk of defects flowing into the next process, dynamically adjust the generated target by using a pre-trained image generation model, combining a region mask image and an assembly stage classification result, so that the monitoring system can adapt to the part form change of different assembly stages, improve the detection accuracy of complex assembly scenes, and through the region mask image and the training loss constraint, the original defects of the workpiece are retained in the generated image, and the newly added information of the process can be accurately reflected, avoiding the influence of historical defects on the monitoring result, and effectively reducing the false positive rate of real-time monitoring for engine production line assembly.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a real-time monitoring method for engine production line assembly. BACKGROUND

[0002] As the core component of power equipment such as automobiles and ships, the assembly quality of the engine directly affects the performance and safety of the product. The traditional engine production line assembly monitoring mainly relies on manual inspection or visual sensor detection. However, the manual inspection method is low in efficiency and difficult to realize real-time monitoring in the whole process, and the recognition accuracy of small defects is insufficient.

[0003] In the visual sensor detection method, the existing technology usually performs production line assembly monitoring based on fixed template matching or simple image comparison, which cannot adapt to the dynamic changes in the engine assembly process, such as the difference in the shape of the parts at different assembly stages, has a high false alarm rate, and the historical defects in the engine assembly process can greatly increase the false alarm rate of process monitoring. For example, scratches on the workpiece and defects allowed by the monitoring personnel in the subsequent process monitoring will still generate warning information because the historical defects and the corresponding positions of the template image are inconsistent.

[0004] Therefore, how to reduce the false alarm rate of real-time monitoring for engine production line assembly has become a problem to be solved. SUMMARY

[0005] In order to solve the above technical problems, the technical scheme adopted by the present application is a real-time monitoring method for engine production line assembly, which comprises the following steps:

[0006] S101, acquiring an assembly image corresponding to each detection point, taking an assembly image corresponding to any detection point as a current image, and taking an assembly image corresponding to the previous detection point of the detection point as a reference image.

[0007] S102, performing foreground extraction on the current image and the reference image respectively to obtain a first image corresponding to the current image and a second image corresponding to the reference image.

[0008] S103, determining a target region corresponding to the first image according to the first image and the second image.

[0009] S104, inputting the second image into a pre-trained image generation model to obtain a generated image.

[0010] S105, acquiring a target region corresponding to the second image as a historical region, and determining a region mask image according to the historical region and the target region corresponding to the first image.

[0011] S106, input the first image and the generated image into the trained assembly stage classification model respectively to obtain a first assembly stage category corresponding to the first image and a second assembly stage category corresponding to the generated image.

[0012] S107, determine a training loss according to the second image, the generated image, the region mask image, the first assembly stage category and the second assembly stage category.

[0013] S108, train the pre-trained image generation model according to the training loss to obtain a trained image generation model, and take the generated image output by the trained image generation model as a target image.

[0014] S109, determine a monitoring and early warning result corresponding to the current image according to the target image and the first image.

[0015] The application further provides an engine production line assembly real-time monitoring system, which comprises:

[0016] An image acquisition module is configured to acquire an assembly image corresponding to each detection point, take the assembly image corresponding to any detection point as a current image, and take the assembly image corresponding to the previous detection point of the detection point as a reference image.

[0017] An image extraction module is configured to perform foreground extraction on the current image and the reference image respectively to obtain a first image corresponding to the current image and a second image corresponding to the reference image.

[0018] A region determination module is configured to determine a target region corresponding to the first image according to the first image and the second image.

[0019] An image generation module is configured to input the second image into a pre-trained image generation model to obtain a generated image.

[0020] An image mask module is configured to acquire a target region corresponding to the second image as a historical region, and determine a region mask image according to the historical region and the target region corresponding to the first image.

[0021] An image classification module is configured to input the first image and the generated image into the trained assembly stage classification model respectively to obtain a first assembly stage category corresponding to the first image and a second assembly stage category corresponding to the generated image.

[0022] A loss calculation module is configured to determine a training loss according to the second image, the generated image, the region mask image, the first assembly stage category and the second assembly stage category.

[0023] a model training module, configured to train the pre-trained image generation model according to the training loss, to obtain a trained image generation model, and to take the generated image output by the trained image generation model as a target image.

[0024] an assembly monitoring module, configured to determine a monitoring and early warning result corresponding to the current image according to the target image and the first image.

[0025] The application further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the engine production line assembly real-time monitoring method when executing the computer program.

[0026] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the engine production line assembly real-time monitoring method.

[0027] The application has at least the following beneficial effects: through time sequence analysis of the current detection point image and the previous detection point image, subtle changes in the assembly process can be captured, missing and misassembly problems can be found in time, the risk of defects flowing into the next process can be significantly reduced, the pre-trained image generation model is used to dynamically adjust the generated target in combination with the region mask image and the assembly stage classification result, the monitoring system can adapt to the part shape changes in different assembly stages, the detection accuracy for complex assembly scenes is improved, through the region mask image and the training loss constraint, the original defects of the workpiece can be retained in the generated image, and the newly added information of the process can be accurately reflected, the influence of historical defects on the monitoring result is avoided, and the false positive rate of the engine production line assembly real-time monitoring is effectively reduced. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0029] Figure 1 A flowchart of an engine production line assembly real-time monitoring method provided by the embodiment one of the application;

[0030] Figure 2 A structural schematic diagram of an engine production line assembly real-time monitoring system provided by the embodiment two of the application. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of the present application.

[0032] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It can be understood that the above-described terms for distinguishing similar objects can be interchanged under appropriate circumstances, so that the present application can also be implemented in other embodiments in addition to the above-described illustrated embodiments or described embodiments. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0033] Embodiment one

[0034] The embodiment one provides a real-time monitoring method for engine production line assembly, such as Figure 1 As shown in the figure, the flowchart of the real-time monitoring method for engine production line assembly provided by the embodiment one of the present application, the real-time monitoring method for engine production line assembly includes the following steps:

[0035] S101, obtaining the assembly image corresponding to each detection point, taking the assembly image corresponding to any detection point as the current image, and taking the assembly image corresponding to the previous detection point of the detection point as the reference image;

[0036] S102, foreground extraction is performed on the current image and the reference image respectively, to obtain the first image corresponding to the current image and the second image corresponding to the reference image;

[0037] S103, determining the target region corresponding to the first image according to the first image and the second image;

[0038] S104, inputting the second image into the pre-trained image generation model to obtain a generated image;

[0039] S105, obtaining the target region corresponding to the second image as a historical region, and determining a region mask image according to the historical region and the target region corresponding to the first image;

[0040] S106, input the first image and the generated image into the trained assembly stage classification model respectively to obtain a first assembly stage category corresponding to the first image and a second assembly stage category corresponding to the generated image;

[0041] S107, determine a training loss according to the second image, the generated image, the region mask image, the first assembly stage category and the second assembly stage category;

[0042] S108, train the pre-trained image generation model according to the training loss to obtain a trained image generation model, and take the generated image output by the trained image generation model as a target image;

[0043] S109, determine a monitoring and early warning result corresponding to the current image according to the target image and the first image.

[0044] Wherein, the detection points can be time points after each process of the engine production line assembly process is completed, the visual sensors used for collecting the assembly images at each detection point can be different, but should all meet the specified view angle collection of the implementer. It should be noted that the present embodiment only takes a single view of engine assembly as the object for processing, for example, all are top views of engine workpieces. The implementer can apply the present embodiment method to production line assembly monitoring of different views, and additional visual sensors need to be deployed for collection.

[0045] The assembly image can be an image collected for the engine workpiece after the corresponding process is completed. The first image and the second image can be images containing only engine workpiece information.

[0046] The target region can be a region adjusted by the corresponding process.

[0047] The image generation model can adopt a U-Net network model. The architecture of the image generation model is not described here. The training data set used by the image generation model during pre-training can take images before a certain process as samples and images after the process as labels.

[0048] The architecture of the assembly stage classification model can include an encoder and a fully connected layer. The input of the encoder is a to-be-classified image, the output of the encoder is a feature vector corresponding to the to-be-classified image, the input of the fully connected layer is the feature vector corresponding to the to-be-classified image, and the output of the fully connected layer is a classification vector mapped from the feature vector. The classification vector is processed by a softmax function to obtain an assembly stage category corresponding to the to-be-classified image. The assembly stage category can be a process of engine assembly.

[0049] Specifically, according to the training loss, the parameters of the pre-trained image generation model are updated by using the gradient descent method until the training loss converges, and a trained image generation model is obtained.

[0050] In a specific embodiment, the foreground extraction of the current image and the reference image respectively obtains a first image corresponding to the current image and a second image corresponding to the reference image, which includes:

[0051] The current image is input into the trained target detection model, and first bounding box information of a first target in the current image is output, the first bounding box information including a first center point, first width and height information, and a first rotation angle;

[0052] The first center point is taken as a first rotation center point, and the current image is rotated according to the first rotation center point and the first rotation angle to obtain a first temporary image;

[0053] According to the first center point and the first width and height information, a first image corresponding to the current image is extracted from the first temporary image;

[0054] The reference image is input into the trained target detection model, and second bounding box information of a second target in the reference image is output, the second bounding box information including a second center point, second width and height information, and a second rotation angle;

[0055] The second center point is taken as a second rotation center point, and the reference image is rotated according to the second rotation center point and the second rotation angle to obtain a second temporary image;

[0056] According to the second center point and the second width and height information, a second image corresponding to the reference image is extracted from the second temporary image.

[0057] Wherein, the target detection model can adopt YOLO model, etc., but the selected target detection model should be able to realize the detection of the rotating target, in this embodiment, YOLOv8 model is adopted as the target detection model, and the architecture and training process of the target detection model are not described here.

[0058] Specifically, the current image and the reference image in this embodiment do not cause the loss of engine workpiece information after rotation by default, so the size of the first temporary image and the second temporary image is not re-determined, and the first image is directly extracted from the first temporary image according to the first positive bounding box determined by the first center point and the first width and height information, and similarly, the second image is directly extracted from the second temporary image according to the second positive bounding box determined by the second center point and the second width and height information.

[0059] It should be noted that, since the visual sensors used in the embodiment are all fixed poses, it can be considered that the collected current image and reference image have been corrected for distortion, and the current image and the reference image collect the same view image of the engine workpiece, and then it can be considered that the sizes of the first image and the second image are approximately the same, and the size alignment of the first image and the second image can be performed by a simple edge zero padding method. The subsequent processing is performed on the premise that the sizes of the first image and the second image are consistent.

[0060] In a specific embodiment, the determining of the target region corresponding to the first image according to the first image and the second image comprises:

[0061] performing difference calculation on the first image and the second image to obtain a difference image;

[0062] performing binaryzation processing on the difference image to obtain a difference binary image, and performing open operation processing on the difference binary image to obtain an intermediate image;

[0063] determining the minimum circumscribed rectangle of all non-zero pixel points in the intermediate image as the target region corresponding to the first image.

[0064] The difference calculation manner of the first image and the second image is to subtract the first image and the second image pixel by pixel and take the absolute value.

[0065] The binaryzation processing can refer to setting the pixel value of a pixel point with a non-zero pixel value to 1, and keeping the pixel value of a pixel point with a zero pixel value as zero.

[0066] The open operation can refer to the operation of first erosion and then dilation on the difference binary image to eliminate noise points.

[0067] Specifically, the minimum circumscribed rectangle can be realized by using the cv2.boundingRect() function in the OpenCV library.

[0068] In a specific embodiment, the determining of the region mask image according to the historical region and the target region corresponding to the first image comprises:

[0069] constructing an initial mask image with the same size as the first image, wherein the pixel value of each pixel point in the initial mask image is zero;

[0070] for any pixel point in the initial mask image, if the pixel point belongs to the target region, keeping the pixel value of the pixel point as zero;

[0071] if the pixel point does not belong to the target region and belongs to the historical region, updating the pixel value of the pixel point to a second preset value;

[0072] if the pixel point does not belong to the target region and does not belong to the history region, updating the pixel value of the pixel point as a third preset value;

[0073] traversing all pixel points in the initial mask image to obtain the region mask image.

[0074] The second preset value can be set as 2, and the third preset value can be set as 10. The implementer can adjust the values of the second preset value and the third preset value according to the actual situation, but needs to ensure that the third preset value is greater than the second preset value, and both the second preset value and the third preset value are greater than zero.

[0075] In a specific embodiment, after step S106 and before step S107, the following steps are further included:

[0076] inputting the second image into the trained assembly stage classification model to obtain a reference assembly stage category corresponding to the second image;

[0077] determining a target assembly stage category corresponding to the reference assembly stage category through a preset lookup table according to the reference assembly stage category;

[0078] if the first assembly stage category is different from the target assembly stage category, generating a first warning information as a monitoring and warning result corresponding to the current image;

[0079] if the first assembly stage category is the same as the target assembly stage category, continuing to execute step S107.

[0080] The reference assembly stage category can represent a process corresponding to the second image, and the preset lookup table includes a next process associated with each process, that is, a target assembly stage category corresponding to each reference assembly stage category.

[0081] Specifically, when the first assembly stage category is different from the target assembly stage category, it can be considered that the first image cannot be accurately corresponded to its process through simple classification, at this time, it is indicated that the engine workpiece in the current image has a large abnormality, and the first warning information can be directly generated as the monitoring and warning result to represent a serious abnormality, without the need for subsequent step processing.

[0082] When the first assembly stage category is the same as the target assembly stage category, it can be considered that the first image can be accurately corresponded to its process through simple classification, at this time, it is indicated that the engine workpiece in the current image does not have a large abnormality, and needs to be processed through subsequent steps to determine whether there is a small abnormality.

[0083] In a specific embodiment, the determining the training loss according to the second image, the generated image, the region mask image, the first assembly stage category and the second assembly stage category comprises:

[0084] For any pixel point, square error calculation is performed according to a pixel value corresponding to the pixel point in the second image and a pixel value corresponding to the pixel point in the generated image, and the square error calculation result is multiplied by a pixel value corresponding to the pixel point in the region mask image to obtain an error term corresponding to the pixel point;

[0085] All error terms corresponding to all pixel points are added, and then the addition result is divided by the number of pixel points to obtain a first sub-loss;

[0086] The second sub-loss is determined according to the first assembly stage category and the second assembly stage category;

[0087] The sum of the first sub-loss and the second sub-loss is taken as the training loss.

[0088] The pixel value corresponding to the pixel point in the region mask image can be regarded as the weight of the square error. It can be known that the calculation of the first sub-loss adopts weighted mean square error calculation.

[0089] Specifically, the pixel value corresponding to the pixel point in the region mask image in the target region is 0, indicating that the target region can be changed to generate an image region of the ideal next process in the target region. The pixel value corresponding to the pixel point in the region mask image in the historical region but not in the target region is a second preset value, which is small, indicating that the historical region can be changed under necessary circumstances, for example, repairing the defect corresponding to the last detection point. The pixel value corresponding to the pixel point in the region mask image not in the target region and not in the historical region is a third preset value, which is large, indicating that other regions are not allowed to change. The second image is input for image generation, rather than directly using the template image as the output. Compared with directly using the template, the embodiment can generate the image based on the second image, retain the original defects of the workpiece, accurately reflect the newly added information of the process, avoid the influence of historical defects on the monitoring result, and reduce the false positive rate of real-time monitoring of engine assembly lines.

[0090] In a specific embodiment, the determining the monitoring and early warning result corresponding to the current image according to the target image and the first image comprises:

[0091] Difference calculation is performed on the target image and the first image to obtain an image to be evaluated;

[0092] The to-be-evaluated image is subjected to binarization processing to obtain a to-be-evaluated binary image;

[0093] The number of non-zero pixel points in the to-be-evaluated binary image is counted to obtain a target number;

[0094] It is determined that the number of pixel points contained in the to-be-evaluated binary image is a reference number.

[0095] The ratio of the target number to the reference number is calculated, and when the ratio is greater than a preset proportion threshold, a second warning information is generated as a monitoring warning result corresponding to the current image.

[0096] The difference calculation manner of the target image and the first image is that the first image and the target image are subtracted pixel by pixel and the absolute value is taken.

[0097] The binarization processing can refer to setting the pixel value of a pixel point with a non-zero pixel value to 1, and keeping the pixel value of a pixel point with a zero pixel value as zero.

[0098] The target number can represent the difference degree of the target image and the first image, and the ratio of the target number to the reference number can represent the normalized result of the difference degree of the target image and the first image.

[0099] Specifically, in the present embodiment, the preset proportion threshold can be set to 0.01, and the implementer can adjust the preset proportion threshold according to actual needs. When the demand for monitoring is strict, a lower preset proportion threshold is set, and when the demand for monitoring is loose, a higher preset proportion threshold is set.

[0100] The second warning information can represent a pending abnormality. When the second warning information is used as a monitoring warning result, a monitoring personnel needs to verify whether there is an abnormality. When the monitoring personnel believes that there is no abnormality, the width and height of the target region of the current image are both set to 0 to avoid affecting subsequent monitoring.

[0101] In the first embodiment, by performing time sequence analysis on the current detection point image and the previous detection point image, subtle changes in the assembly process can be captured, and problems such as missing assembly and misassembly can be found in time, thereby significantly reducing the risk of defects flowing into the next process. By using a pre-trained image generation model, combining a region mask image and an assembly stage classification result to dynamically adjust the generated target, the monitoring system can adapt to the shape changes of parts in different assembly stages, improve the detection accuracy for complex assembly scenes, and through the region mask image and the training loss constraint, the original defects of the workpiece are retained in the generated image, and the newly added information of the process is accurately reflected, thereby avoiding the influence of historical defects on the monitoring result and effectively reducing the false positive rate of real-time monitoring for engine production line assembly.

[0102] Embodiment Two

[0103] The embodiment two provides a real-time monitoring system for engine production line assembly, as shown in Figure 2 The real-time monitoring system for engine production line assembly provided by the embodiment two of the application has the structure as shown in the figure, and comprises:

[0104] The image acquisition module 201 is configured to acquire an assembly image corresponding to each detection point, take an assembly image corresponding to any detection point as a current image, and take an assembly image corresponding to a previous detection point of the detection point as a reference image.

[0105] The image extraction module 202 is configured to perform foreground extraction on the current image and the reference image respectively to obtain a first image corresponding to the current image and a second image corresponding to the reference image.

[0106] The region determination module 203 is configured to determine a target region corresponding to the first image according to the first image and the second image.

[0107] The image generation module 204 is configured to input the second image into a pre-trained image generation model to obtain a generated image.

[0108] The image mask module 205 is configured to acquire a target region corresponding to the second image as a historical region, and determine a region mask image according to the historical region and the target region corresponding to the first image.

[0109] The image classification module 206 is configured to input the first image and the generated image into the trained assembly stage classification model respectively to obtain a first assembly stage category corresponding to the first image and a second assembly stage category corresponding to the generated image.

[0110] The loss calculation module 207 is configured to determine a training loss according to the second image, the generated image, the region mask image, the first assembly stage category and the second assembly stage category.

[0111] The model training module 208 is configured to train the pre-trained image generation model according to the training loss to obtain a trained image generation model, and take a generated image output by the trained image generation model as a target image.

[0112] The assembly monitoring module 209 is configured to determine a monitoring and early warning result corresponding to the current image according to the target image and the first image.

[0113] It should be noted that the specific definition of the real-time monitoring system for engine production line assembly can refer to the definition of the real-time monitoring method for engine production line assembly in the above, which will not be repeated here. The information interaction, execution process and other contents between the above modules, since the same concept based on the method embodiment of the application, its specific function and the technical effect brought about, specific can refer to the method embodiment part, this place will not be repeated.

[0114] Embodiment three

[0115] This embodiment three provides a kind of computer equipment, which can be server. The computer equipment can include processor, memory, network interface and database connected by system bus. Wherein, the processor of the computer equipment is used to provide calculation and control capability. The memory of the computer equipment includes nonvolatile storage medium, internal memory. The nonvolatile storage medium stores operating system, computer program and database. The internal memory provides the environment for the operation of operating system and computer program in nonvolatile storage medium. The network interface of the computer equipment is used to communicate with external terminal through network connection. The computer program is executed by processor to realize a kind of real-time monitoring method for engine production line assembly.

[0116] Embodiment four

[0117] This embodiment four provides a kind of computer readable storage medium, and the computer readable storage medium stores computer program, and the computer program is executed by processor to realize the real-time monitoring method for engine production line assembly in above-mentioned embodiment, to avoid repetition, here will not be repeated. Or, the computer program is executed by processor to realize the function of each module / unit in this embodiment of the above real-time monitoring system for engine production line assembly, to avoid repetition, here will not be repeated.

[0118] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchronous Link) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0119] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the system is divided into different functional units or modules to complete all or part of the above-described functions.

[0120] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Although the present application has been disclosed as above with a preferred embodiment, it is not intended to limit the present application. Any skilled person in the art can make some changes or modifications to the above-mentioned disclosed technical content to make equivalent embodiments with equivalent changes, but as long as it does not deviate from the technical solution of the present application, any simple modification, equivalent change and modification of the above-mentioned embodiments according to the technical essence of the present application are still within the scope of the technical solution of the present application.

Claims

1. A real-time monitoring method for engine production line assembly, characterized in that: The real-time monitoring method for engine production line assembly includes the following steps: S101, obtaining assembly images corresponding to respective detection points, taking the assembly image corresponding to any detection point as the current image, and taking the assembly image corresponding to the previous detection point of the detection point as the reference image; S102, performing foreground extraction on the current image and the reference image respectively to obtain a first image corresponding to the current image and a second image corresponding to the reference image; S103: Determine a target area corresponding to the first image based on the first image and the second image, wherein determining the target area corresponding to the first image based on the first image and the second image includes: performing a difference calculation on the first image and the second image to obtain a difference image; performing a binarization process on the differential image to obtain a differential binary image, and performing an opening operation process on the differential binary image to obtain an intermediate image; Determine a minimum bounding rectangle of all non-zero pixels in the intermediate image as a target area corresponding to the first image; S104, inputting the second image into a pre-trained image generation model to obtain a generated image; S105, acquiring a target area corresponding to the second image as a historical area, and determining a regional mask image according to the historical area and the target area corresponding to the first image; S106, inputting the first image and the generated image into the trained assembly stage classification model respectively to obtain a first assembly stage category corresponding to the first image and a second assembly stage category corresponding to the generated image; S107, determining a training loss based on the second image, the generated image, the region mask image, the first assembly stage category, and the second assembly stage category; S108, training the pre-trained image generation model according to the training loss to obtain a trained image generation model, and using a generated image output by the trained image generation model as a target image; S109: Determine a monitoring warning result corresponding to the current image according to the target image and the first image.

2. The real-time monitoring method for engine production line assembly according to claim 1, characterized in that: The performing foreground extraction on the current image and the reference image respectively to obtain a first image corresponding to the current image and a second image corresponding to the reference image includes: Input the current image into a trained object detection model, and output first bounding box information of a first object in the current image, where the first bounding box information includes a first center point, first width and height information, and a first rotation angle; Taking the first center point as a first rotation center point, rotating the current image according to the first rotation center point and the first rotation angle to obtain a first temporary image; extracting a first image corresponding to the current image from the first temporary image according to the first center point and the first width and height information; Inputting the reference image into the trained object detection model, and outputting second bounding box information of a second object in the reference image, where the second bounding box information includes a second center point, second width and height information, and a second rotation angle; Taking the second center point as a second rotation center point, rotating the reference image according to the second rotation center point and the second rotation angle to obtain a second temporary image; A second image corresponding to the reference image is extracted from the second temporary image according to the second center point and the second width and height information.

3. The real-time monitoring method for engine production line assembly according to claim 1, characterized in that: The determining of a region mask image according to the historical region and the target region corresponding to the first image includes: constructing an initial mask image having the same size as the first image, wherein the pixel value of each pixel in the initial mask image is zero; For any pixel point in the initial mask image, if the pixel point belongs to the target area, the pixel value of the pixel point is kept as zero; If the pixel point does not belong to the target area but belongs to the historical area, updating the pixel value of the pixel point to a second preset value; If the pixel point does not belong to the target area and does not belong to the historical area, updating the pixel value of the pixel point to a third preset value; All pixels in the initial mask image are traversed to obtain the regional mask image.

4. The real-time monitoring method for engine production line assembly according to claim 1, characterized in that: After step S106 and before step S107, the following steps are also included: Inputting the second image into the trained assembly stage classification model to obtain a reference assembly stage category corresponding to the second image; According to the reference assembly stage category, determining the target assembly stage category corresponding to the reference assembly stage category through a preset lookup table; If the first assembly stage category is different from the target assembly stage category, generating first warning information as a monitoring warning result corresponding to the current image; If the first assembly stage category is the same as the target assembly stage category, then continue to step S107.

5. The real-time monitoring method for engine production line assembly according to claim 1, characterized in that: The determining of the training loss according to the second image, the generated image, the region mask image, the first assembly stage category, and the second assembly stage category includes: For any pixel point, perform square error calculation based on the pixel value corresponding to the pixel point in the second image and the pixel value corresponding to the pixel point in the generated image, and multiply the square error calculation result by the pixel value corresponding to the pixel point in the region mask image to obtain an error term corresponding to the pixel point; Traverse all pixels, add up the error terms corresponding to all pixels, and divide the sum by the number of pixels to get the first sub-loss; determining a second sub-loss based on the first assembly stage category and the second assembly stage category; The sum of the first sub-loss and the second sub-loss is used as the training loss.

6. The real-time monitoring method for engine production line assembly according to claim 1, characterized in that: The determining, based on the target image and the first image, a monitoring and warning result corresponding to the current image includes: Performing a difference calculation on the target image and the first image to obtain an image to be evaluated; Performing binarization processing on the image to be evaluated to obtain a binary image to be evaluated; Counting the number of non-zero pixels in the binary image to be evaluated to obtain a target number; Determining the number of pixels contained in the binary image to be evaluated as a reference number; The ratio of the target quantity to the reference quantity is calculated, and when the ratio is greater than a preset ratio threshold, a second warning information is generated as a monitoring warning result corresponding to the current image.

7. A real-time monitoring system for engine production line assembly, characterized in that: The real-time monitoring system for engine production line assembly includes: An image acquisition module is used to acquire assembly images corresponding to each detection point, with the assembly image corresponding to any detection point being used as the current image and the assembly image corresponding to the previous detection point being used as the reference image; an image extraction module, configured to perform foreground extraction on the current image and the reference image respectively, to obtain a first image corresponding to the current image and a second image corresponding to the reference image; A region determination module is configured to determine a target region corresponding to the first image based on the first image and the second image, wherein determining the target region corresponding to the first image based on the first image and the second image includes: performing a difference calculation on the first image and the second image to obtain a difference image; performing a binarization process on the differential image to obtain a differential binary image, and performing an opening operation process on the differential binary image to obtain an intermediate image; Determine a minimum bounding rectangle of all non-zero pixels in the intermediate image as a target area corresponding to the first image; an image generation module, configured to input the second image into a pre-trained image generation model to obtain a generated image; an image mask module, configured to obtain a target area corresponding to the second image as a historical area, and determine a regional mask image based on the historical area and the target area corresponding to the first image; an image classification module, configured to input the first image and the generated image into the trained assembly stage classification model, respectively, to obtain a first assembly stage category corresponding to the first image and a second assembly stage category corresponding to the generated image; a loss calculation module, configured to determine a training loss based on the second image, the generated image, the region mask image, the first assembly stage category, and the second assembly stage category; A model training module is used to train the pre-trained image generation model according to the training loss to obtain a trained image generation model, and use the generated image output by the trained image generation model as the target image; The assembly monitoring module is used to determine the monitoring warning result corresponding to the current image based on the target image and the first image.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the real-time monitoring method for engine production line assembly according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the real-time monitoring method for engine production line assembly according to any one of claims 1 to 6 is implemented.

Images