An image processing method, a related model training method, and related devices

Abstract

The present application discloses an image processing method, a related model training method, and related devices. The image processing method includes: obtaining a plurality of original images; using the plurality of original images to form a target processed image; using a trained image processing model to perform duplicate removal processing on the target processed image to obtain a first spliced image. The image processing model training method includes: obtaining a plurality of sample images and obtaining a reference spliced image, using the plurality of sample images to form a sample processed image; using the image processing model to perform duplicate removal processing on the sample processed image to obtain a sample spliced image; using the difference between the reference spliced image and the sample spliced image to adjust the network parameters of the image processing model. The above solution can improve the splicing efficiency and quality of images.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and in particular, to an image processing method, a related model training method, and related devices. Background Art

[0002] Currently, image processing technologies have been widely applied in daily life and various industries. For example, an object to be detected can be photographed, and then relevant information of the object can be detected using the photographed image. Often, for some large objects, it is difficult to completely photograph them with a single image. At this time, multiple images of different parts of the object need to be taken, and then these multiple images are stitched together to obtain a complete image of the object. Image stitching generally involves aligning a series of spatially overlapping images and constructing a seamless and high-definition image, which has always been an area of interest in computer graphics and machine vision.

[0003] The effect of image stitching often affects the subsequent processing effect of the stitched image. Therefore, how to achieve high-quality and efficient image stitching is particularly important. Summary of the Invention

[0004] The main technical problem to be solved by this application is to provide an image processing method, a related model training method, and related devices, which can improve the efficiency and quality of image stitching.

[0005] To solve the above technical problem, a technical solution adopted by this application is: to provide an image processing method; the method includes: obtaining multiple original images, where the multiple original images are obtained by photographing different parts of a target object; using the multiple original images to form a target processing image; using a trained image processing model to perform duplicate removal processing on the target processing image to obtain a first stitched image, where the duplicate removal processing includes removing overlapping regions in different original images.

[0006] Among them, using the multiple original images to form a target processing image includes: arranging the multiple original images in a preset order to form a target processing image with a first preset shape and a preset size.

[0007] Among them, the first preset shape is a square.

[0008] Among them, arranging the multiple original images in a preset order to form a target processing image with a first preset shape and a preset size includes: if the multiple original images are not sufficient to form the preset size, arranging and combining the multiple original images in a preset order, and filling the remaining part that is less than the preset size after combination with pixel values to obtain a target processing image with a first preset shape and a preset size, where the filled pixel values are not the pixel values of the target parts of the original images that belong to the target object.

[0009] Among them, the image processing model is a convolutional autoencoder model.

[0010] Among them, before using the trained image processing model to perform duplicate removal processing on the target processed image to obtain the first stitched image, the image processing model is trained, including the following steps: obtaining multiple sample images and obtaining a reference stitched image, where the multiple sample images are obtained by photographing different parts of the sample object, the sample object and the target object are of the same category, and the reference stitched image represents the stitching standard of the multiple sample images; using the multiple sample images to form a sample processed image; using the image processing model to perform duplicate removal processing on the sample processed image to obtain a sample stitched image; using the difference between the reference stitched image and the sample stitched image to adjust the network parameters of the image processing model. Among them, the sample processed image, the sample stitched image, and the reference stitched image are all of the first preset shape.

[0011] Among them, obtaining the reference stitched image includes: obtaining the original object image input by the user, where the original object image is obtained by the user stitching multiple sample images, cropping the original object image into several first sub-images, and using the several first sub-images to form the reference stitched image.

[0012] Among them, after using the trained image processing model to perform duplicate removal processing on the target processed image to obtain the first stitched image, it further includes: reorganizing the first stitched image of the first preset shape to form a second stitched image of the second preset shape; performing target recognition on the second stitched image to obtain the recognition information of the target part of the target object in the second stitched image.

[0013] Among them, reorganizing the first stitched image of the first preset shape to form a second stitched image of the second preset shape includes: based on the size of the first stitched image, cropping the first stitched image of the first preset shape into several second sub-images; sequentially combining the several second sub-images to obtain the second stitched image of the second preset shape.

[0014] Among them, the recognition information of the target part of the target object includes at least one of the following: the number of target parts, the position of the target parts, and the size information of the target parts.

[0015] Among them, performing target recognition on the second stitched image to obtain the recognition information of the target part of the target object in the second stitched image includes: using the Hough transform method to perform target recognition on the second stitched image to obtain the recognition information of the target part of the target object in the second stitched image.

[0016] Among them, the target object is an automotive longitudinal beam, the original image contains the target part of the target object, and the target part is a hole.

[0017] Before composing the target processed image from multiple original images, it further includes: converting the multiple original images into black and white images or grayscale images.

[0018] To solve the above technical problems, another technical solution adopted by this application is: providing a training method for an image processing model; including the following steps: obtaining multiple sample images and obtaining a reference stitching image, wherein the multiple sample images are obtained by photographing different parts of a sample object, and the reference stitching image represents the stitching standard of the multiple sample images; using the multiple sample images to compose a sample processed image; using the image processing model to perform duplicate removal processing on the sample processed image to obtain a sample stitching image; using the difference between the reference stitching image and the sample stitching image to adjust the network parameters of the image processing model.

[0019] Among them, the sample processed image, the sample stitching image, and the reference stitching image are all of a first preset shape.

[0020] Among them, using the multiple sample images to compose a sample processed image includes: arranging the multiple sample images in a preset order to compose a sample processed image of a first preset shape and a preset size.

[0021] Among them, obtaining the reference stitching image includes: obtaining an original object image input by a user, wherein the original object image is obtained by the user stitching the multiple sample images; cropping the original object image into several first sub-images, and using the several first sub-images to compose the reference stitching image.

[0022] To solve the above technical problems, another technical solution adopted by this application is: providing an image processing device, which includes an acquisition module, a composition module, and a processing module. The acquisition module is used to acquire multiple original images, wherein the multiple original images are obtained by photographing different parts of a target object; the composition module is used to use the multiple original images to compose a target processed image; the processing module is used to use a trained image processing model to perform duplicate removal processing on the target processed image to obtain a first stitching image, wherein the duplicate removal processing includes removing overlapping regions in different original images.

[0023] To solve the above technical problems, another technical solution adopted in this application is: to provide a training device for an image processing model, which includes a sample acquisition module, a sample composition module, a sample processing module, and an adjustment module. The sample acquisition module is used to acquire multiple sample images and a reference stitching image, where the multiple sample images are obtained by photographing different parts of a sample object, and the reference stitching image represents the stitching standard of the multiple sample images; the sample composition module is used to compose a sample processing image using the multiple sample images; the sample processing module is used to perform duplicate removal processing on the sample processing image using the image processing model to obtain a sample stitching image; the adjustment module is used to adjust the network parameters of the image processing model using the difference between the reference stitching image and the sample stitching image.

[0024] To solve the above technical problems, another technical solution adopted in this application is: to provide an image processing device, including a memory and a processor coupled to each other, where the memory stores program instructions; the processor is used to execute the program instructions stored in the memory to implement the above image processing method, and / or, the training method of the image processing model.

[0025] To solve the above technical problems, another technical solution adopted in this application is: to provide a computer-readable storage medium, which is used to store program instructions that can be executed to implement the above image processing method, and / or, implement the training method of the image processing model.

[0026] In the above solution, by first composing a target processing image from multiple original images of different parts of the target object, and then using the trained image processing model to perform duplicate removal processing on the target processing image, which is equivalent to obtaining a stitching image stitched from multiple original images. Since the solution of this application directly combines images first and then uses the network model to perform duplicate removal on the combined image to achieve image stitching, and compared with performing registration through a large number of feature point matches between each original image and then performing projective transformation on the original images to achieve image stitching, the solution of this application directly uses the image processing model to achieve image stitching, which can reduce the processing computing power and processing time during the stitching process, improve the stitching efficiency, and since the image processing model has been trained, it can achieve a certain accurate duplicate removal and stitching effect, so the quality of the stitched image can be improved.

[0027] In addition, by using multiple sample images to form a sample processing image, and then using an image processing model to perform duplicate removal processing on the sample processing image to obtain a sample stitching image, the network parameters of the image processing model are adjusted by comparing the differences between the reference stitching image and the obtained sample stitching image. By training the image processing model in this way, it can be ensured that the image obtained after the duplicate removal processing by the image processing model is equivalent to the ideal stitching image, so as to achieve a certain accurate duplicate removal and stitching effect, and thus improve the quality of the stitching image. Moreover, during the training process of the image processing model, it automatically learns using the sample processing image data, which means that the trained image processing model can realize the automatic stitching processing of images, significantly improving the image stitching efficiency. Description of the Drawings

[0028] Figure 1 It is a schematic flowchart of an embodiment of the image processing method provided by the present application;

[0029] Figure 2a It is a schematic flowchart of an embodiment of the image processing model training method provided by the present application;

[0030] Figure 2b It is a schematic diagram of image changes in the process of obtaining a reference stitching image in an embodiment of the image processing model training method provided by the present application;

[0031] Figure 3a It is a partial schematic flowchart of another embodiment of the image processing method provided by the present application;

[0032] Figure 3b It is a schematic diagram of image changes in the image processing process in another embodiment of the image processing method provided by the present application;

[0033] Figure 4 It is a schematic framework diagram of an embodiment of the image processing device provided by the present application;

[0034] Figure 5 It is a schematic framework diagram of an embodiment of the training device of the image processing model provided by the present application;

[0035] Figure 6 It is a schematic framework diagram of an embodiment of the image processing device provided by the present application;

[0036] Figure 7 It is a schematic framework diagram of an embodiment of the computer-readable storage medium provided by the present application. Detailed Embodiments

[0037] To make the objectives, technical solutions and effects of the present application clearer and more definite, the present application will be further described in detail below with reference to the accompanying drawings and by way of examples.

[0038] It should be noted that if the descriptions such as "first" and "second" are involved in the embodiments of the present application, the descriptions of "first", "second", etc. are only for descriptive purposes and should not be construed as indicating or implying their relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions between various embodiments may be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement them. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0039] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of an embodiment of the image processing method provided by the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 1 the process sequence shown. As Figure 1 shown, this embodiment includes:

[0040] Step S110: Obtain multiple original images.

[0041] The method of this embodiment is used to perform image processing on multiple original images through an image processing model to obtain an image that is automatically stitched and the overlapping areas are removed.

[0042] The multiple original images described herein are obtained by photographing different parts of the target object. For example, at least 2 cameras are placed on a shooting rack to photograph different parts of the target object to obtain multiple original images, and the number of cameras can be adjusted according to the size of the target object. In some embodiments, considering that the stitched image can include all parts of the target object, the number of cameras is based on the principle of being able to achieve panoramic shooting of the target object. Another example is that a single camera can be used to perform moving shooting on the target object to obtain multiple original images. Specifically, for example, the camera starts shooting from one end of the target object and moves the camera during the shooting process to continuously shoot the target object, so as to obtain multiple original images corresponding to different parts of the target object. It can be understood that the acquisition method of the original images can be but is not limited to any one of the above methods.

[0043] Among them, the target object can be any object. In this embodiment, the target object is an object whose full view cannot be shown by a single image. Different parts of the target object refer to different regions of the target object. The complete target object or a part of the target object can be formed by combining different parts of the target object. The target object described in this article can be, but is not limited to, large objects such as automotive longitudinal beams, bridges, and buildings. The original image contains the target part of the target object. For example, the target part can be any component existing in the target object, such as a hole, a pier, etc. In a specific application scenario, the target object is an automotive longitudinal beam and the target part is a hole.

[0044] Step S120: Use multiple original images to form a target processed image.

[0045] In this embodiment, the multiple original images are arranged and combined in a preset order to form a target processed image. Specifically, the target processed image can be an image with a first preset shape and a preset size. The first preset shape described in this article can be, but is not limited to, a square, a rectangle, etc. Among them, the preset order mainly considers the shooting parts of the multiple original images, and tries to make the adjacent original images of the captured adjacent parts adjacent. In addition, considering to meet the preset size, it may also be necessary to form the multiple original images into multiple rows and columns. For example, taking the size of one original image as a unit, the preset size is 3*3. If the multiple original images correspond to the parts of the target object from left to right, the preset order can be to arrange three original images from left to right in the first row first, and then arrange three original images from left to right in the second row. In this way, each row is sorted from left to right in turn, or arrange three original images from left to right in the first row first, and then arrange three original images from right to left in the second row. Taking the arrangement method of every two rows as a cycle, and sorting in this way in turn; if the multiple original images correspond to the parts of the target object from top to bottom, the preset order is to arrange three original images from top to bottom in the first column first, and then arrange three original images from top to bottom in the second column. In this way, sort in turn, or arrange three original images from top to bottom in the first column first, and then arrange three original images from bottom to top in the second column. Taking the arrangement method of every two columns as a cycle, and sorting in this way in turn. It can be understood that the preset order can be, but is not limited to, any of the above arrangement orders, and no specific limitation is made here.

[0046] Among them, the preset size can be set according to the actual situation. In some embodiments, it has been preset when training the following image processing model, so that when training and subsequently applying the image processing model, the size of the input image is unified. Of course, if the image processing model has low dependence on the input image size, the preset size does not need to be unified when training and subsequently applying the image processing model, that is, images of different sizes can be input. In addition, the size of the preset size is usually greater than or equal to the total size of multiple original images to ensure that all contents of the multiple original images can be included.

[0047] When multiple original images are insufficient to form the preset size, the multiple original images can be arranged and combined in a preset order first, and the remaining part that is insufficient for the preset size after combination is filled with pixel values to obtain a target processed image with a first preset shape and a preset size. Among them, the filled pixel values can be, but are not limited to, any pixel values such as black, white, or color. In the case where the stitched image is used to identify the target part in the target object, it is necessary to fill pixel values that are different from the target part of the target object, so that there is a difference between the pixel values of the target part of the target object and the filled pixel values, that is, the filled pixel values are not the pixel values of the target part belonging to the original image. In some embodiments, in order to better avoid the influence of the filled area on the recognition of the target part, the difference between the filled pixel values and the pixel values of the target part can be relatively large, so that the pixel value difference between the two is greater than the preset pixel threshold. In addition, the filled pixel values can refer to the pixel values of the background area in the original image except for the target part. For example, the filled pixel values can be set to the same pixel values as the background area. Figure 3b For example, the number of multiple original images is 5. Taking the size of one original image as a unit, the preset size is 3*3. Therefore, the 5 original images are arranged and combined in a preset order to obtain an image combination area 31, and the remaining area 32 that is insufficient for the preset size is filled with pixel values, that is, the target processed image with the preset size is composed of area 31 and area 32. Among them, the target part is a hole in the target object, and the original image is a black-and-white image (in order to better show each area in the figure, the black is not shown as black in the figure, but is replaced by a grid). The target part hole in the original image is white, and the background area is black. Therefore, the pixel values in the filled area 32 can also be set to the pixel values corresponding to black. Thus, the filled area 32 is regarded as the background part of the original image and does not affect the recognition of the target part.

[0048] In other embodiments, in order to improve the recognition degree of the image, facilitate edge detection of the image or recognition of some parts of the target object, multiple original pictures can be converted in the manner of step S120, such as converted into a black-and-white image or a grayscale image, or can not be converted according to the actual situation, which is not specifically limited here.

[0049] Step S130: Use the trained image processing model to perform duplicate removal processing on the target processed image to obtain a first stitched image.

[0050] In this embodiment, use the trained image processing model to perform duplicate removal processing on the target processed image to obtain a first stitched image. The shape of the first stitched image may be the same as or different from that of the input target processed image. Among them, the duplicate removal processing includes removing overlapping regions in different original images. The trained image processing model described herein is a model that can perform image duplicate removal processing on the target processed image. The image processing model may be, but is not limited to, a convolutional autoencoder. The convolutional autoencoder is an unsupervised neural network model that uses convolutional and deconvolutional modules in the bottleneck structure to reproduce the input image through the output layer and extract useful feature information from the middle layer of the network. It can be understood that the image processing model may also be other network structures capable of reproducing images, and the network structure of the image processing model is not specifically limited herein.

[0051] Among them, the training process of the image processing model can refer to Figure 2a the relevant description of. In some embodiments, before step S120 (including before step S110), the image processing method of the present application may include Figure 2a the steps in the relevant embodiments.

[0052] In the above embodiment, by first forming a target processed image from multiple original images of different parts of the target object, and then using the trained image processing model to perform duplicate removal processing on the target processed image, it is equivalent to obtaining a stitched image obtained by stitching multiple original images. Since the solution of the present application first directly combines images and then uses a network model to perform duplicate removal on the combined image to achieve image stitching, and compared with performing registration by matching a large number of feature points between the original images and then performing projective transformation on the original images to achieve image stitching, the solution of the present application directly uses the image processing model to achieve image stitching, which can reduce the processing computing power and processing time in the stitching process, improve the stitching efficiency, and since the image processing model has been trained, it can achieve a certain accurate duplicate removal and stitching effect, so the quality of the stitched image can be improved.

[0053] Please refer to Figure 2a , Figure 2a which is a schematic flowchart of an embodiment of the image processing model training method provided by the present application. In this embodiment, the image processing model training process includes:

[0054] Step S210: Obtain multiple sample images and obtain a reference stitched image.

[0055] Multiple sample images are obtained by photographing different parts of a sample object. The photographing method can be, but is not limited to, any one of the photographing methods described in step S110. In some embodiments, the multiple obtained sample images can be processed to be converted into black-and-white images or grayscale images to improve the recognition of the images and facilitate subsequent edge detection of the images. Or, according to actual situations, the conversion may not be performed, and no specific limitation is made here.

[0056] Among them, the sample object is of the same type as the target object described in step S110. For example, they are both vehicle longitudinal beams or both bridges, etc. Different parts of the sample object refer to different regions of the sample object. The combination of different parts of the sample object can form a complete sample object or a part of the sample object.

[0057] The reference stitching image represents the stitching standard of multiple sample images, that is, it can be considered as the ideal situation after stitching multiple sample images. In some embodiments, the reference stitching image is determined by the user. For example, obtaining the reference stitching image includes: obtaining the original object image input by the user, where the original object image is obtained by the user stitching the multiple sample images; cropping the original object image into several first sub-images, and using the several first sub-images to form the reference stitching image. Among them, the cropping of the original object image can be automatically performed by the device or in response to the user's operation. Specifically, in combination with Figure 2bAn example is given for the process of obtaining the reference stitched image. First, the user stitches the multiple sample images according to the parts of the target object corresponding to the multiple sample images, so that the part situation of the sample object in the stitched original object image is consistent with the part situation of the sample object in the actual environment, that is, the original object image is similar to the image directly taken of the whole target object. After obtaining the original object image, considering that both the input image and the output image of the image processing model are of the first preset shape, in order to facilitate the subsequent comparison of the differences between the original object image and the input image, the original object image is cropped into several first sub-images 21, and each first sub-image is of the same size. Then, referring to the way of forming the target processing image from the original images, several first sub-images are formed into a reference stitched image. For example, several first sub-images are formed into a reference stitched image of the first preset shape and specified size in a preset order. If the several first sub-images are not enough to form the specified size, the several first sub-images are arranged and combined in a preset order, and the remaining part that is less than the specified size after the combination is filled with pixel values to obtain several first sub-images of the first preset shape and specified size. In some embodiments, the pixel values filled in the reference stitched image may not be the pixel values of the target parts of the sample object in the sample image, and the selection of the filled pixel values can refer to the relevant description of filling pixel values in the above target processing image. By obtaining the reference stitched image as the stitching standard for the multiple sample images, the training result of the subsequent image processing model is further determined.

[0058] Step S220: Use multiple sample images to form a sample processing image.

[0059] By arranging and combining multiple sample images in a preset order to form a sample processing image. Specifically, the sample processing image can be of the first preset shape and preset size, and the first preset shape can be but is not limited to a square or a rectangle. In this embodiment, the shape and size of the sample processing image are the same as those of the above target processing image, but in some embodiments, if the shape and size of the sample processing image and the target processing image can also be different. For example, during the training process, sample processing images of different sizes are input for the image so that the model can accurately remove duplicates for input images of different sizes. Therefore, during the subsequent use of the model, a target processing model with an unrestricted size can be used. The same applies to the image size. If sample processing images of different shapes are input for the image during the training process so that the model can accurately remove duplicates for input images of different shapes, then during the subsequent use of the model, a target processing model with an unrestricted shape can be used.

[0060] For the specific description of using multiple sample images to form a sample processing image, reference can be made to the description of using multiple original images to form a target processing image in step S120 above, and details will not be elaborated here.

[0061] Step S230: Use the image processing model to perform duplicate removal on the sample processed image to obtain a sample spliced image.

[0062] Regarding the duplicate removal process of the image processing model and the introduction of the related network structure, reference can be made to the description in step S130 above, which will not be elaborated here.

[0063] In some embodiments, the sample processed image, the sample spliced image, and the reference spliced image are all of a first preset shape.

[0064] Step S240: Use the difference between the reference spliced image and the sample spliced image to adjust the network parameters of the image processing model.

[0065] The reference spliced image serves as the splicing standard for multiple sample images. During the training process, the loss of the current image processing model can be obtained based on the difference between the reference spliced image and the sample spliced image, and then the network parameters of the image processing model can be adjusted using the loss. When the difference between the reference spliced image and the sample spliced image reaches the preset requirement, and / or the number of training times reaches the preset requirement, it indicates that the training of the image processing model is completed. The preset requirement can be determined according to the actual situation. In some specific applications, the preset requirement can be based on not affecting the subsequent recognition of the target part of the image, which will not be specifically limited here.

[0066] In this embodiment, by training the image processing model, an image processing model capable of automatically splicing and removing duplicates of images can be obtained, which is convenient for directly performing splicing and duplicate removal processing on the target object subsequently. Moreover, through training, it can be ensured that the image obtained after duplicate removal by the image processing model is comparable to the ideal spliced image, achieving a certain accurate duplicate removal and splicing effect, so the quality of the spliced image can be improved. In addition, during the training process of the image processing model, it automatically learns using the sample processed image data, which means that the trained image processing model can realize the automatic splicing processing of images, significantly improving the splicing efficiency of images.

[0067] In some embodiments, during the training process of the image model, images taken under different lighting conditions or with different cameras can be selected as sample images. Thus, the trained image processing model can be made not to be affected by lighting conditions, the stability of the acquisition camera, etc. during the image splicing and duplicate removal process, thereby further improving the overall quality of the spliced image.

[0068] Please refer to Figure 3a , Figure 3a which is a partial flowchart of another embodiment of the image processing method provided by this application. In this embodiment, after step S130, it further includes using the first spliced image to realize the recognition of the target part of the target object. Specifically, such asFigure 3a As shown, after step S130, the embodiment further includes the following steps:

[0069] Step S310: reorganizing the first stitched images of the first preset shape into a second stitched image of the second preset shape.

[0070] In this embodiment, in order to facilitate the subsequent detection of the target part, the first stitched image can be reorganized into a second preset shape. Specifically, based on the size of the first stitched image, the first stitched image of the first preset shape can be cropped into a plurality of second sub-images, and then the obtained plurality of second sub-images are sequentially combined to obtain a second stitched image of the second preset shape. Among them, the size of each second sub-image is the same, or the height of each second sub-image is the same, and the width can be the same or different. The specific size of the second sub-image can be determined according to the size of the first stitched image and the cropping method. In some application scenarios, the target object is a long strip, and each second sub-image corresponds to a section of the long strip. The order of combining the second sub-images can be reverse stitching in the preset order used when combining the target processed images in step S120. For example, step S120 is to combine multiple original images sorted from left to right in a manner that each row is arranged from left to right to obtain a target processed image, then take out each second sub-image in a manner that each row is arranged from left to right, and arrange and combine the taken out second sub-images from left to right to obtain a second stitched image. It can be understood that the second spliced image is a continuous image after multiple original images are spliced and deduplicated, which can reflect the overall appearance of the target object and can be understood as the target object image. The second preset shape can be different from the image shape of the first preset shape. For example, the first preset shape is a square, and the second preset shape is a rectangle. The rectangle is obtained by arranging the plurality of second sub-images in a row. Of course, the second preset shape can also be a plurality of second sub-images arranged in more than two rows, or a square. The specific situation can be determined according to the actual situation and is not specifically limited here.

[0071] Combination Figure 3bFor example, in this embodiment, the target object is a longitudinal beam. First, five segmented longitudinal beam images (i.e., original images) are obtained by photographing the longitudinal beam in segments. The five segmented longitudinal beam images are combined to form a square target processed image. Among them, taking the size of one original image as a unit, the size of the target processed image is 3*3, and the areas other than the five longitudinal beam images in the target processed image are filled with pixel values. The target processed image is processed using an image processing model to obtain a first stitched image that is also square. Among them, after the first stitched image is de-duplicated, its size is smaller than that of the target processed image. Based on the size of the first stitched image, the first stitched image is cropped into four second sub-images, and each second sub-image corresponds to a section of the longitudinal beam. Then the four second sub-images are sequentially stitched to obtain a rectangular second stitched image, that is, the second stitched image is equivalent to including the entire picture of the longitudinal beam.

[0072] Step S320: Perform target recognition on the second stitched image to obtain recognition information about the target part of the target object in the second stitched image.

[0073] In one embodiment, the target object can be but is not limited to an automotive longitudinal beam, a bridge, etc., and the target part can be but is not limited to a hole in an automotive longitudinal beam, a pier, etc. The recognition information of the target part includes at least one of the following: the number of target parts, the position of the target part, and the size information of the target part. In a specific embodiment, the target object is a vehicle longitudinal beam, and the target part is a round hole in the vehicle longitudinal beam. The recognition information of the target part includes the number of round holes, the center position information of each round hole, and the aperture information of each round hole, etc.

[0074] In some embodiments, this step S320 may specifically include: performing target recognition on the second stitched image by using the Hough transform method to obtain the recognition information about the target part of the target object in the second stitched image. Of course, other recognition methods can also be used, such as using a neural network for target recognition, which is not specifically limited here.

[0075] Please refer to Figure 4 , Figure 4 is a schematic framework diagram of an embodiment of an image processing device provided in the present application. In this embodiment, the image processing device 40 includes: an acquisition module 41, a composition module 42, and a processing module 43. The acquisition module 41 is used to acquire multiple original images, where the multiple original images are obtained by photographing different parts of the target object; the composition module 42 is used to form a target processed image by using the multiple original images; the processing module 43 is used to perform de-duplication processing on the target processed image by using a trained image processing model to obtain a first stitched image, where the de-duplication processing includes removing overlapping regions in different original images.

[0076] In some embodiments, the above-mentioned composition module 42 is used to compose a target processed image by using multiple original images, including: arranging the multiple original images in a preset order to compose a target processed image with a first preset shape and a preset size. Optionally, the first preset shape is a square.

[0077] In some embodiments, the composition module 42 is used to arrange multiple original images in a preset order to compose a target processed image with a first preset shape and a preset size. Specifically, if the multiple original images are not enough to compose the preset size, the multiple original images are arranged and combined in a preset order, and the remaining part that is less than the preset size after combination is filled with pixel values to obtain a target processed image with a first preset shape and a preset size, where the filled pixel values are not the pixel values of the target parts belonging to the target object in the original images.

[0078] In some embodiments, the image processing device 40 further includes a conversion module, which is used to convert the multiple original images into black and white images or grayscale images before the composition module 42 composes a target processed image by using the multiple original images.

[0079] In some embodiments, the processing module 43 is used to perform duplicate removal processing on the target processed image by using a trained image processing model to obtain a first spliced image, where the duplicate removal processing includes removing overlapping regions in different original images.

[0080] In some embodiments, after obtaining the first spliced image, the composition module 42 is used to reorganize the first spliced image with a first preset shape into a second spliced image with a second preset shape. Specifically, based on the size of the first spliced image, the first spliced image with a first preset shape is cropped into several second sub-images, and the several second sub-images are sequentially combined to obtain a second spliced image with a second preset shape.

[0081] In some embodiments, the image processing device 40 further includes an identification module, which is used to perform target identification on the second spliced image after the composition module 42 reorganizes the first spliced image into a second spliced image with a second preset shape to obtain identification information about the target parts of the target object in the second spliced image. In one embodiment, the target object is an automotive longitudinal beam, the original images contain the target parts of the target object, the target parts are holes, and the identification information about the target parts of the target object includes at least one of the following: the number of target parts, the position of the target parts, and the size information of the target parts. The Hough transform method can be used to perform target identification on the second spliced image to obtain the identification information about the target parts of the target object in the second spliced image.

[0082] It should be noted that the device in this embodiment can execute the steps in the above method. For the detailed description of related content, please refer to the above method part and will not be repeated here.

[0083] Please refer to Figure 5 , Figure 5 which is a schematic framework diagram of an embodiment of a training device for an image processing model provided by this application. In this embodiment, the image processing device 50 includes: a sample acquisition module 51, a sample composition module 52, a sample processing module 53, and an adjustment module 54, which are used to train the image model. The sample acquisition module 51 is used to acquire multiple sample images and a reference stitching image; among them, the multiple sample images are obtained by photographing different parts of a sample object, and the reference stitching image represents the stitching standard of the multiple sample images; the sample composition module 52 is used to compose a sample processing image by using the multiple sample images; the sample processing module 53 is used to perform duplicate removal processing on the sample processing image by using the image processing model to obtain a sample stitching image; the adjustment module 54 is used to adjust the network parameters of the image processing model by using the difference between the reference stitching image and the sample stitching image.

[0084] In some embodiments, the sample composition module 52 is used to compose a sample processing image from multiple sample images, specifically including: arranging the multiple sample images in a preset order to form a sample processing image with a first preset shape and a preset size.

[0085] In some embodiments, the sample acquisition module 51 is used to acquire a reference stitching image, including: acquiring an original object image input by a user, where the original object image is obtained by the user stitching multiple sample images using the sample composition module 52; cropping the original object image into several first sub-images, and using the sample composition module 52 to compose the several first sub-images into a reference stitching image.

[0086] In some embodiments, the sample processing image, the sample stitching image, and the reference stitching image are all of the first preset shape.

[0087] It should be noted that the device in this embodiment can execute the steps in the above method. For a detailed description of related content, please refer to the above method part and will not be repeated here.

[0088] Please refer to Figure 6 , Figure 6 which is a schematic framework diagram of an embodiment of an image processing device provided by this application. In this embodiment, the image processing device 60 includes a memory 61 and a processor 62.

[0089] The processor 62 can also be referred to as a CPU (Central Processing Unit). The processor 62 may be an integrated circuit chip with the ability to process signals. The processor 62 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor 62 can also be any conventional processor 62, etc.

[0090] The memory 61 in the image processing device 60 is used to store program instructions required for the operation of the processor 62.

[0091] The processor 62 is used to execute program instructions to implement the methods provided by any one of the above-described embodiments of the image processing method and the image model training method in the present application and any non-conflicting combination.

[0092] Please refer to Figure 7 , Figure 7 is a schematic framework diagram of a computer-readable storage medium provided by the present application. The computer-readable storage medium 70 of the embodiments of the present application stores program instructions 71, and when the program instructions 71 are executed, the methods provided by any one of the image processing method and the image model training method in the present application and any non-conflicting combination are implemented. Among them, the program instructions 71 can form a program file and be stored in the above-mentioned computer-readable storage medium 70 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) can execute all or part of the steps of the methods of various embodiments of the present application. The foregoing computer-readable storage medium 70 includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, or terminal devices such as computers, servers, mobile phones, and tablets.

[0093] In the above solution, an image stitching is realized by using an image processing model, the frequency of abnormal distortion of the stitched image is reduced, and the overall quality of the fused image is improved. In addition, subsequent part recognition based on the stitched image obtained by the image processing model can improve the visual recognition ability of the target part and effectively improve the robustness of the recognition system to external interference.

[0094] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0095] The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments. Their similarities or resemblances can be referred to each other. For the sake of brevity, they will not be elaborated herein.

[0096] In several embodiments provided in this application, it should be understood that the disclosed methods and apparatuses can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the apparatuses or units can be in electrical, mechanical or other forms.

[0097] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0098] In addition, in each embodiment of this application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0099] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in each embodiment of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0100] The above are only the embodiments of the present application, and do not thus limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall similarly be included within the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that, Including: Obtaining a plurality of original images, where the plurality of original images are partial images obtained by photographing different parts of a target object; Arranging the plurality of original images in rows and columns in a preset order to form a target processed image of a first preset shape; Using a trained image processing model to perform duplicate removal processing on the target processed image to obtain a first stitched image, where the duplicate removal processing includes removing overlapping regions in different original images; Based on the size of the first stitched image, cropping the first stitched image of the first preset shape into a plurality of second sub-images, and the heights of the second sub-images are the same; Sequentially combining the plurality of second sub-images to obtain a second stitched image of a second preset shape; the second stitched image is a full-view image of the target object, and the first preset shape and the second preset shape are different; Performing target recognition on the second stitched image to obtain recognition information about the target part of the target object in the second stitched image.

2. The method according to claim 1, wherein The first preset shape is a square, and the size of the target processed image is a preset size; And / or, arranging the plurality of original images in rows and columns in a preset order to form the target processed image of the first preset shape includes: If the plurality of original images are not enough to form the preset size, arranging and combining the plurality of original images in a preset order, and filling the remaining part that is less than the preset size after the combination with pixel values to obtain the target processed image of the first preset shape and the preset size, where the filled pixel values are not the pixel values of the target part of the target object in the original image.

3. The method according to claim 1, characterized in that The image processing model is a convolutional autoencoder model; And / or, before using the trained image processing model to perform duplicate removal processing on the target processed image to obtain a first stitched image, training the image processing model includes: Obtaining a plurality of sample images and obtaining a reference stitched image, where the plurality of sample images are obtained by photographing different parts of a sample object, the sample object is of the same type as the target object, and the reference stitched image represents the stitching standard of the plurality of sample images; Using the plurality of sample images to form a sample processed image; Using the image processing model to perform duplicate removal processing on the sample processed image to obtain a sample stitched image; Adjusting the network parameters of the image processing model using the difference between the reference stitched image and the sample stitched image.

4. The method according to claim 3, wherein The sample processed image, the sample stitched image, and the reference stitched image are all of the first preset shape; And / or, obtaining the reference stitched image includes: Obtaining an original object image input by a user, where the original object image is obtained by the user stitching the plurality of sample images; Cropping the original object image into a plurality of first sub-images, and using the plurality of first sub-images to form the reference stitched image.

5. The method according to claim 1, characterized in that, The recognition information about the target part of the target object includes at least one of the following: the number of the target parts, the positions of the target parts, and the size information of the target parts; And / or, performing object recognition on the second stitched image to obtain recognition information about a target part of the target object in the second stitched image, including: Performing object recognition on the second stitched image by using a Hough transform method to obtain recognition information about a target part of the target object in the second stitched image.

6. The method according to claim 1, wherein The target object is a vehicle longitudinal beam, the original image includes a target part of the target object, and the target part is a hole; And / or, before using the multiple original images to form a target processing image, the method further includes: Converting the multiple original images into black and white images or grayscale images.

7. A training method for an image processing model, characterized in that Including: Obtaining multiple sample images and obtaining a reference stitched image, where the multiple sample images are partial images obtained by photographing different parts of a sample object, and the reference stitched image represents the stitching standard of the multiple sample images; Arranging the multiple sample images in rows and columns in a preset order to form a sample processing image with a first preset shape; Performing duplicate removal processing on the sample processing image by using the image processing model to obtain a sample stitched image; Adjusting network parameters of the image processing model by using the difference between the reference stitched image and the sample stitched image; The image processing model with adjusted network parameters is used to perform duplicate removal processing on a target processing image to obtain a first stitched image, where the target processing image is an image with a first preset shape formed by arranging multiple original images in rows and columns in a preset order, the multiple original images are partial images obtained by photographing different parts of a target object, and the duplicate removal processing includes removing overlapping regions in different original images; The first stitched image is used to be cropped into a plurality of second sub-images, and the heights of the second sub-images are the same; the plurality of second sub-images are used to be sequentially combined to obtain a second stitched image with a second preset shape, the second stitched image is a full-view image of the target object, and the shapes of the first stitched image and the second stitched image are different; the second stitched image is used for object recognition to obtain recognition information about a target part of the target object in the second stitched image.

8. The method according to claim 7, wherein The sample processing image, the sample stitched image, and the reference stitched image are all in the first preset shape; And / or, obtaining the reference stitched image includes: Obtaining an original object image input by a user, where the original object image is obtained by the user stitching the multiple sample images; Cropping the original object image into a plurality of first sub-images and using the plurality of first sub-images to form the reference stitched image.

9. An image processing apparatus, characterized in that, The device includes: An obtaining module, configured to obtain multiple original images, where the multiple original images are partial images obtained by photographing different parts of a target object; A composing module, configured to arrange the multiple original images in rows and columns in a preset order to form a target processing image with a first preset shape; A processing module, configured to perform duplicate removal processing on the target processed image by using a trained image processing model to obtain a first spliced image, wherein the duplicate removal processing includes removing overlapping regions in different original images; A recombination module, configured to crop the first spliced image of the first preset shape into a plurality of second sub-images based on the size of the first spliced image, where the heights of the second sub-images are the same; combine the plurality of second sub-images in sequence to obtain a second spliced image of a second preset shape; the second spliced image is a full-view image of the target object, and the first preset shape and the second preset shape are different; perform target recognition on the second spliced image to obtain recognition information about the target part of the target object in the second spliced image.

10. A training device for an image processing model, characterized in that The device includes: A sample acquisition module, configured to acquire a plurality of sample images and acquire a reference spliced image, wherein the plurality of sample images are partial images obtained by photographing different parts of a sample object, and the reference spliced image represents the splicing standard of the plurality of sample images; A sample composition module, configured to arrange the plurality of sample images in rows and columns in a preset order to form a sample processed image of a first preset shape; A sample processing module, configured to perform duplicate removal processing on the sample processed image by using the image processing model to obtain a sample spliced image; An adjustment module, configured to adjust the network parameters of the image processing model by using the difference between the reference spliced image and the sample spliced image; The image processing model with adjusted network parameters is used to perform duplicate removal processing on a target processed image to obtain a first spliced image, wherein the target processed image is an image of a first preset shape formed by arranging a plurality of original images in rows and columns in a preset order, the plurality of original images are partial images obtained by photographing different parts of a target object, and the duplicate removal processing includes removing overlapping regions in different original images; The first spliced image is used to be cropped into a plurality of second sub-images, where the heights of the second sub-images are the same; the plurality of second sub-images are used to be combined in sequence to obtain a second spliced image of a second preset shape; the second spliced image is a full-view image of the target object, and the shapes of the first spliced image and the second spliced image are different; the second spliced image is used for target recognition to obtain recognition information about the target part of the target object in the second spliced image.

11. An image processing apparatus, characterized in that, Including a memory and a processor that are coupled to each other, The memory stores program instructions; The processor is configured to execute the program instructions stored in the memory to implement the method according to any one of claims 1-6, and / or, implement the method according to any one of claims 7-8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium is configured to store program instructions, and the program instructions can be executed to implement the method according to any one of claims 1-6, and / or, implement the method according to any one of claims 7-8.

Images