Image processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment and a storage medium. The method comprises the steps of collecting background image data and target object image data; calculating frame difference data between the target object image data and the background image data; extracting position information of the target object based on the frame difference data; determining the coordinate position and size of the target object based on the position information; based on the coordinate position and the size, extracting an effective area containing the target object from the target object image data, and generating effective area image data; and splicing the effective area image data to obtain spliced image data. According to the image splicing scheme provided by the invention, the image splicing efficiency and accuracy can be improved, excessive man-made interference is not needed, the dependence on computing resources is reduced, automatic image acquisition and splicing are realized, and the problems of low efficiency, poor splicing effect, low automation degree and the like in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of digital image processing technology, image stitching technology has become a key technology in the field of machine vision. This technology can stitch and fuse multiple images according to certain rules into a large-viewing-field and high-resolution image, and is widely used in many fields such as industrial inspection, medical imaging, photography, and digital image processing. In these applications, image stitching technology can provide a wider field of view and higher resolution, thereby enhancing the ability to measure the size, shape matching, and defect detection of large-size objects.

[0003] However, existing image stitching technologies face a series of challenges. First, traditional image stitching methods rely on overlapping areas and feature point matching between images, which requires that there must be sufficient intersection areas and rich texture feature points between images. This requirement limits the application of stitching technology on small targets or images with sparse feature points. Secondly, feature point matching algorithms usually have huge computational complexity and slow processing speed, making it difficult to meet application scenarios with high real-time requirements. In addition, these methods often require precise calibration of the camera in advance to obtain parameters such as camera intrinsics, which not only increases the complexity of the operation, but also increases the requirements for operator skills, making it difficult to achieve fully automated image acquisition and stitching processes. It can be seen that the existing technology has problems such as low efficiency, insufficient accuracy, and low degree of automation.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the invention

[0005] In response to the above technical problems, the present application provides an image processing method, device, electronic device and storage medium to solve the problems of low efficiency, insufficient accuracy and low degree of automation in the prior art.

[0006] In order to solve the above technical problems, the present application provides an image processing method, comprising the following steps:

[0007] Collecting background image data and target image data;

[0008] Calculating frame difference data between the target image data and the background image data;

[0009] Extracting the position information of the target object based on the frame difference data;

[0010] Determine the coordinate position and size of the target object based on the position information;

[0011] extracting a valid area including the target object from the target object image data based on the coordinate position and the size, and generating valid area image data;

[0012] The effective area image data are spliced ​​to obtain spliced ​​image data.

[0013] Furthermore, in some embodiments of the present application, the collecting of background image data and target object image data includes:

[0014] Collecting background image data of the target object by a camera device;

[0015] The image data corresponding to the target object is collected by the camera device starting from the origin of the control platform until it rotates 360 degrees to obtain the corresponding image data of the target object.

[0016] Furthermore, in some embodiments of the present application, the collecting of background image data and target object image data includes:

[0017] Collecting background image data of several different surfaces corresponding to the target object by a camera device;

[0018] The camera device collects the target object image every 90 degrees from the origin of the control platform to obtain the corresponding target object image data.

[0019] Further, in some embodiments of the present application, determining the coordinate position and size of the target object based on the position information includes:

[0020] Calculate the histograms corresponding to the target object in the horizontal direction and the vertical direction based on the position information;

[0021] Based on a preset histogram threshold, determining the starting position and the ending position of the target object in the histograms in the horizontal direction and the vertical direction respectively;

[0022] Based on the starting position and the ending position, the coordinate position and the size of the target object are calculated.

[0023] Further, in some embodiments of the present application, extracting a valid area including the target object from the target object image data based on the coordinate position and the size to generate the valid area image data includes:

[0024] Based on the coordinate position and size of the target object, determine the coordinates corresponding to the effective area containing the target object;

[0025] Based on the coordinates corresponding to the effective area, corresponding effective area image data is extracted from the target image data.

[0026] Furthermore, in some embodiments of the present application, the collecting of background image data and target object image data further includes:

[0027] The control platform is controlled to rotate according to a preset rotation rule, so as to collect background image data and target object image data through a camera device on the control platform.

[0028] Accordingly, the present application provides an image stitching device, comprising:

[0029] A data acquisition module, used for acquiring background image data and target image data;

[0030] A frame difference calculation module, used for calculating frame difference data between the target image data and the background image data;

[0031] A position extraction module, used to extract the position information of the target object based on the frame difference data;

[0032] A target object determination module, used to determine the coordinate position and size of the target object based on the position information;

[0033] A region extraction module, configured to extract a valid region including the target object from the target object image data based on the coordinate position and the size, and generate valid region image data;

[0034] The image stitching module is used to stitch the image data in the effective area to obtain stitched image data.

[0035] Further, in some embodiments of the present application, the data acquisition module is also used to respond to the acquisition instruction and generate an image acquisition instruction for the control platform;

[0036] Based on the image acquisition instruction, the control platform is controlled to rotate according to a preset rotation rule, so as to acquire background image data and target object image data through the camera device on the control platform.

[0037] The present application also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image processing method described above when executing the program.

[0038] The present application also provides a storage medium storing a computer program that can be loaded by a processor and execute the image processing method as described above.

[0039] Implementing the embodiments of the present application has the following beneficial effects:

[0040] As described above, the present application provides an image processing method, device, electronic device and storage medium, and the image processing method includes: collecting background image data and target image data; calculating frame difference data between the target image data and the background image data; extracting the position information of the target based on the frame difference data; determining the coordinate position and size of the target based on the position information; extracting the effective area containing the target from the target image data based on the coordinate position and size, and generating effective area image data; splicing the effective area image data to obtain spliced ​​image data. The image splicing scheme provided by the present application can accurately determine the position and size of the target through precise frame difference calculation and histogram analysis, thereby improving the accuracy of subsequent image splicing; through automated image acquisition and processing procedures, it reduces manual intervention, improves the efficiency of image splicing, and reduces dependence on computing resources, thereby realizing automated image acquisition and splicing. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0042] Figure 1 It is a schematic diagram of an application scenario of the image processing method provided in an embodiment of the present application;

[0043] Figure 2 is a flowchart of an image processing method provided in an embodiment of the present application;

[0044] Figure 3 is a structural schematic diagram of an image stitching device provided in an embodiment of the present application;

[0045] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0046] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The above-mentioned drawings have shown clear embodiments of this application, which will be described in more detail later. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0047] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0048] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0049] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0051] At present, image stitching mainly includes image registration and image fusion. Image registration is the basis of image fusion. In addition, the computational complexity of image registration algorithms is generally very large, and the accuracy of registration is uncertain. The development of image stitching technology depends largely on the innovation of image registration technology. Early image registration technology mainly used point matching methods, which are slow, low-precision, and often require manual selection of initial matching points. They cannot adapt to the fusion of large amounts of data and the current fully automatic generation pipeline.

[0052] However, current stitching technologies all have certain problems. First, the two stitched images need to have enough intersecting areas and enough matching texture feature points. It is difficult to achieve a better stitching effect for small targets. Secondly, there are problems with speed and the number of feature points when using feature point matching calculations: using feature points to determine the overlapping areas between images also has the characteristics of complex calculation process and slow speed, which is difficult for some fields with high real-time requirements; in addition, if the number of feature points is small, it may lead to matching failure, which is also limited for practical applications. Thirdly, these methods all require the camera to be calibrated in advance to obtain parameters including at least the camera intrinsic parameters. The accuracy of the object parameters and the requirements for operators are relatively high, and it is difficult to achieve a fully automatic process.

[0053] In order to solve the above technical problems, the present application provides an image processing method, device, electronic device and storage medium.

[0054] The image stitching device can be integrated into an electronic device, which can be a smart phone, a tablet computer, a laptop or a desktop computer, but is not limited thereto. The electronic device can be directly or indirectly connected to the server through wired or wireless communication. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. This application does not limit this.

[0055] See also Figure 1 , Figure 1 FIG. 1 is an application environment diagram of an image processing method in an embodiment. Figure 1 , the image processing method can be applied to an image stitching system. The image stitching system may include a terminal 110 and a server 120. The terminal 110 and the server 120 are connected via a network. The terminal 110 may be a desktop terminal or a mobile terminal. The mobile terminal may be at least one of a mobile phone, a tablet computer, a laptop computer, etc. The server 120 may be implemented as an independent server or a server cluster composed of multiple servers. The terminal 110 is used to collect background image data and target image data; calculate the frame difference data between the target image data and the background image data; extract the position information of the target based on the frame difference data; determine the coordinate position and size of the target based on the position information; extract the effective area containing the target from the target image data based on the coordinate position and size, and generate effective area image data; stitch the effective area image data to obtain stitched image data.

[0056] It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.

[0057] The present application provides an image processing method, including: collecting background image data and target image data; calculating frame difference data between the target image data and the background image data; extracting position information of the target based on the frame difference data; determining the coordinate position and size of the target based on the position information; extracting a valid area containing the target from the target image data based on the coordinate position and size, and generating valid area image data; and splicing the valid area image data to obtain spliced ​​image data.

[0058] See also Figure 2 , Figure 2 : is a flowchart of an image processing method provided in an embodiment of the present application. The image processing method provided in this embodiment may specifically include the following steps:

[0059] S1. Collect background image data and target image data;

[0060] Specifically, for step S1, first, background image data of the environment where the target object is located needs to be collected. This step ensures that a pure background image can be obtained in the absence of the target object for subsequent frame difference calculation. Next, image data containing the target object is collected, which can be a static image or a video sequence, depending on the shape of the target object, or depending on whether the target object moves or rotates.

[0061] S2. Calculate the frame difference data between the target image data and the background image data;

[0062] Specifically, for step S2, by comparing the target image data with the background image data, the difference between the target image data and the background image data, i.e., frame difference data, is calculated. This step can highlight the difference between the target and the background, providing a basis for subsequent target position extraction.

[0063] S3. Extracting the position information of the target object based on the frame difference data;

[0064] Specifically, in step S3, the frame difference data is used to identify the outline and position of the current target object, for example, by performing threshold judgment and processing on the frame difference data to distinguish the target object from the background.

[0065] S4. Determine the coordinate position and size of the target object based on the position information;

[0066] Specifically, for step S4, the precise coordinate position and size of the target in the image are further determined based on the extracted target position information, for example, by analyzing the frame difference data using a histogram method to determine the distribution of the target in the horizontal and vertical directions, thereby accurately determining the coordinate position and size of the target.

[0067] S5. Based on the coordinate position and size, extracting the effective area containing the target object from the target image data, generating effective area image data;

[0068] Specifically, in step S5, according to the coordinate position and size of the target object, a valid area including the target object is cropped from the original image data to ensure that only the image data of the target object is involved in the stitching process.

[0069] S6. stitching the effective area image data to obtain stitched image data;

[0070] Specifically, for step S6, finally, all extracted valid area image data are spliced ​​together, and image alignment and fusion technology is used to generate a complete and seamlessly spliced ​​image data.

[0071] It can be seen that the image processing method provided in this embodiment can accurately determine the position and size of the target object through precise frame difference calculation and histogram analysis, thereby improving the accuracy of image stitching; through automated image acquisition and processing procedures, manual intervention is reduced and the efficiency of image stitching is improved. It is not only applicable to static images, but also can process dynamic video data, making it applicable to a wider range of application scenarios. And because this method reduces the dependence on high computing units and reduces hardware costs, the image stitching technology is more economical and practical. Through image alignment and fusion technology, high-quality stitched images can be generated, stitching traces are reduced, and the visual effect of the image is improved. It can be seen that this embodiment has broad application prospects in industrial detection, medical imaging, photography art and other fields, and can meet the needs of image stitching in different fields.

[0072] Furthermore, in some embodiments, step S1 “collecting background image data and target object image data” may specifically include:

[0073] Collecting background image data of the target object through a camera device;

[0074] The image data corresponding to the target object is collected by the camera device starting from the origin of the control platform until it rotates 360° to obtain the corresponding image data of the target object.

[0075] Specifically, for cylindrical targets, this embodiment first requires that the camera device be installed at a predetermined position and adjusted to a predetermined angle to ensure that the cylindrical target can be fully covered and the image can be captured from the best viewing angle. Before the cylindrical target is placed or enters the field of view of the camera device, a background image is captured, and the background image will be used as a reference for subsequent frame difference calculations to identify and distinguish the target from the background. When the cylindrical target is placed on the control platform, starting from the origin of the platform, as the platform rotates, the video image data of the target is continuously collected until the platform completes a 360° rotation, ensuring that the image of the target is captured from all angles, providing comprehensive data for subsequent image stitching.

[0076] This embodiment can ensure that the image of the cylindrical target is captured from all possible angles through 360° continuous image acquisition, so that the surface of the target can be fully covered during stitching. Since the all-round image of the target is acquired, the stitched image can more accurately reflect the actual shape and characteristics of the target, thereby improving the quality of the stitched image. By controlling the rotation of the platform and the automated acquisition of the camera device, manual operation is reduced and the accuracy and repeatability of the acquisition process are improved. The automated acquisition process reduces the possibility of human operation errors and improves the reliability of image stitching.

[0077] Furthermore, in some embodiments, step S1 “collecting background image data and target object image data” may specifically include:

[0078] Collecting background image data of several different surfaces corresponding to the target object through a camera device;

[0079] The camera device collects the target image every 90 degrees from the origin of the control platform to obtain the corresponding target image data.

[0080] Specifically, for the rectangular target object, the present embodiment firstly needs to install the camera device at a predetermined position and adjust it to a predetermined angle to ensure that each face of the rectangular target object can be optimally captured. Then, the camera device is used to collect background image data of several different faces corresponding to the rectangular target object. Before the target object is not placed or enters the field of view of the camera device, the background images of each face of the rectangular target object are collected. These background images will serve as the basis for subsequent frame difference calculations to identify and distinguish the target object from the background. When the rectangular target object is placed on the control platform, starting from the origin of the platform, an image of the target object is collected every 90 degrees of rotation until a 360° rotation is completed. This step ensures that images of the target object are captured from all angles, providing comprehensive data for subsequent image stitching.

[0081] This embodiment can ensure that images of the rectangular target object are captured from all major angles by collecting images during rotation at 90-degree intervals, so that the surface of the target object can be fully covered during stitching; since multi-angle images of the target object are collected, the stitched image can more accurately reflect the actual shape and characteristics of the target object, thereby improving the quality of the stitched image; by controlling the rotation of the platform and the automated acquisition of the camera device, manual operations are reduced, thereby improving the accuracy and repeatability of the acquisition process. The possibility of human operation errors is reduced through the automated acquisition process, thereby improving the reliability of image stitching. Since this embodiment does not rely on complex hardware equipment, such as high-precision cameras or a large number of computing units, the cost is reduced.

[0082] Further, in some embodiments, step S4 “determining the coordinate position and size of the target object based on the position information” may specifically include:

[0083] S41. Calculate the histogram corresponding to the target in the horizontal and vertical directions based on the position information;

[0084] S42. Based on a preset histogram threshold, determine the starting position and the ending position of the target object in the histogram in the horizontal direction and the vertical direction respectively;

[0085] S43. Calculate the coordinate position and size of the target object based on the start position and the end position.

[0086] Specifically, for step S4, first, the target position information extracted from the frame difference data is used to calculate the histograms in the horizontal and vertical directions respectively. Among them, the histogram is a statistical tool for displaying the distribution of pixel values ​​in an image. In the present embodiment, the histogram is particularly used to identify the edges and contours of the target. Then, one or more thresholds are set to identify the starting and ending positions of the target in the histogram. Among them, the threshold is usually determined based on the characteristics of the target and the background noise level to ensure accuracy and robustness. Once the starting and ending positions of the target in the histogram are determined, the coordinate position (e.g., upper left corner coordinates) and size (e.g., width and height) of the target in the image can be calculated, thereby improving the accuracy of the subsequent extraction of the effective area of ​​the target from the image.

[0087] This embodiment can accurately locate the position of the target object in the image through the histogram method and threshold setting, which is crucial for subsequent image processing and analysis. Automated size measurement reduces the need for manual measurement and improves efficiency and accuracy, especially when a large amount of image data needs to be processed. Accurate target position and size information helps to correctly align the individual image fragments during the image stitching process, thereby improving the accuracy and quality of the final stitched image. Through preset thresholds and histogram analysis, it has a certain robustness to different lighting conditions and background noise, and can work stably in a changing environment.

[0088] Further, in some embodiments, step S5 “extracting a valid area containing the target object from the target object image data based on the coordinate position and size to generate valid area image data” may specifically include:

[0089] S51. Based on the coordinate position and size of the target object, determine the coordinates corresponding to the valid area containing the target object;

[0090] S52. Based on the coordinates corresponding to the effective area, extract the corresponding effective area image data from the target image data.

[0091] Specifically, for step S5, the boundary coordinates of the effective area containing the target object in the image are determined according to the coordinate position and size of the target object calculated in step S4, for example, the upper left corner coordinates (x, y) of the target object, and the lower right corner coordinates based on the width and height of the target object. Using the above coordinates, the effective area containing the target object is cropped from the original target object image data. The cropped effective area image data will be used in the subsequent image stitching process to ensure that only the image part containing the target object is involved in the stitching process, eliminating unnecessary background interference.

[0092] This embodiment can ensure that the cropped image data accurately contains the target object by accurately determining the effective area coordinates of the target object, thereby reducing errors in the stitching process; the cropped effective area image data is directly used for stitching, which reduces the time and computing resources for processing irrelevant background data and improves the efficiency of the stitching process; by eliminating background interference, the quality of the stitched image is improved, and the display of the target object is clearer and more accurate; because only the image data containing the target object is processed, data redundancy is reduced, making storage and processing more efficient. The automated image cropping and data extraction process reduces human intervention and improves the degree of automation of the entire image stitching process. The automated target object image data extraction process reduces the possibility of human operating errors and improves the reliability of image stitching.

[0093] Further, in some embodiments, step S6 of “obtaining stitched image data based on stitching the effective area image data” may specifically include:

[0094] S61. Perform image alignment processing on the valid image data;

[0095] S62. Use an image fusion algorithm to stitch the valid image data after the alignment process to obtain stitched image data.

[0096] Specifically, for step S6, before stitching, the extracted valid area image data needs to be preprocessed, including image alignment, to ensure that all image segments can be accurately matched when stitching and reduce stitching errors. First, feature point detection and matching techniques, such as SIFT, SURF or ORB, are used to identify and correspond to key points in different image segments to achieve accurate alignment. Based on the feature point matching results, the transformation matrix (such as affine transformation, perspective transformation) is calculated, and these transformations are applied to adjust the image to make it geometrically consistent. Then, image fusion techniques, such as multi-band fusion, Poisson fusion or Laplace pyramid fusion, are used to seamlessly stitch the aligned images together. In order to reduce stitching marks, the color and lighting of the stitching area need to be adjusted to ensure a smooth transition of color and brightness of the image at the stitching. In the edge area of ​​the image stitching, a smoothing algorithm (such as Gaussian blur or median filtering) is applied to reduce obvious seams and discontinuities.

[0097] This embodiment can achieve seamless or almost seamless image stitching and provide a continuous and consistent visual effect through precise image alignment and advanced image fusion technology; color and lighting adjustment and edge smoothing improve the overall quality of the stitched image, reduce stitching marks, and make the stitched image more natural and realistic; feature point matching and image fusion technology help to retain details in the image, especially at the edges of the target and areas with rich textures; the automated image alignment and stitching process reduces manual intervention and improves the automation level of the entire image stitching process; the automated image alignment and stitching process significantly improves the efficiency of image processing, especially when a large amount of image data needs to be processed.

[0098] Furthermore, in some embodiments, step S1 “collecting background image data and target object image data” may further include:

[0099] The control platform is controlled to rotate according to a preset rotation rule so as to collect background image data and target object image data through a camera device on the control platform.

[0100] Specifically, for step S1, the control platform rotates according to the image acquisition instruction and a predetermined rotation rule (for example, uniform rotation or rotation at specific angle intervals) to ensure that the image of the target object is acquired from different angles. The rotation of the platform can be precisely synchronized with the shooting of the camera device to ensure that the image of the target object can be accurately captured at each key angle during the rotation process. In the process of controlling the rotation of the platform, the camera device acquires image data of the background and the target object at a specific angle or continuously according to the acquisition instruction. The acquired data needs to be synchronized with the rotation angle and position of the platform so that the viewing angle corresponding to each image segment can be accurately identified during subsequent processing.

[0101] This embodiment reduces the need for manual operation and improves the efficiency and accuracy of image acquisition through automated acquisition instructions and platform control; accurately controls the platform rotation and camera shooting to ensure the consistency of the quality and viewing angle of the acquired image data, which is beneficial to subsequent image processing and stitching; through precise image acquisition and rotation control, the final stitched image can better reflect the actual situation of the target object, thereby improving the quality of the stitched image.

[0102] In order to make it easier to understand the image processing method provided by the present application, examples are given below for objects of different shapes.

[0103] For cylindrical objects, the existing stitching algorithm mainly calculates the feature points of the image and matches the image according to the feature points. However, the volume calculation of the feature points is large, the efficiency is low, and the stitching effect is poor for small objects and small image overlap areas. The algorithm provided in this embodiment can achieve efficient and accurate stitching of images. The specific process is as follows:

[0104] (1) Data collection of the background image: After the installation is completed and the position and angle of the camera are debugged, the background image collection should be started to obtain the background image, where Ib(i,j), Ib(i,j) represents the pixel value of the background image, and i and j represent the coordinate points of the background image.

[0105] (2) After the background image is acquired, the image data acquisition of the cylindrical target begins. For the cylindrical target, the image data acquisition mainly refers to the acquisition of video image data, which starts from the origin of the control platform and ends after the platform rotates 360 degrees. The video image data is V(f,i,j), which represents the data of a single frame image. Here, f represents the low-level frame image, i and j represent the pixel values ​​of the coordinate points of the f frame image at positions i and j. The current position of the target object needs to be calculated for each frame, so as to calculate the frame difference data between the target image data and the background image data, FDiffe(f,i,j)=V(f,i,j)-Ib(i,j). When the frame difference data is greater than the threshold Thre, it means that there is a difference between the target image data and the background image data, and the extracted difference is the position of the cylindrical target object being measured.

[0106] (3) Based on the above frame difference data, the position of the cylindrical target can be calculated. The current position of the cylindrical target, that is, the position of the pixel, needs to obtain the data of the outer rectangle to determine the position of the pixel. The calculation method is to obtain it through the histogram. First, obtain the horizontal histogram Histi(f,i)+=(FDiffe(f,i,j)>Thre?0,1) and the vertical histogram Histj(f,j)+=(FDiffe(f,i,j)>Thre?0,1); the thresholds of the histogram are Thre_Histi (horizontal threshold) and Thre_Histj (vertical threshold). Based on the horizontal The horizontal direction histogram threshold and the vertical direction histogram threshold are used to determine the starting position and ending position of the target object in the horizontal direction and vertical direction respectively. For example, all histograms are traversed to find the starting position start_Histi (horizontal direction), start_Histj (vertical direction) greater than the threshold and the ending position end_Histi (horizontal direction), end_Histj (vertical direction) greater than the threshold, so as to obtain the position coordinates of the cylindrical target object x=start_Histi, y=start_Histj, w=end_Histi-start_Histi, h=end_Histj-start_Histj.

[0107] According to the above calculation, the diameter size (unit: pixel number) D=w of the cylindrical target in the image and the height size (unit: pixel number) H=h of the cylindrical target in the image can be obtained. According to the current data, the effective area in the image can be extracted. The width of the effective area is Yw (can be set by the user), and the position coordinates of the effective area are Yx=x+(w-Yw) / 2, Yy=y, Yw and Yh=h.

[0108] (4) Extracting effective area image data from each frame of the video according to the above-mentioned coordinate diagram, splicing the effective area image data, and finally outputting the spliced ​​image data.

[0109] Among them, for cylindrical targets, a control method for the control platform and a method for collecting video image data are also provided. The main process is as follows: first, rotate the control platform in the opposite direction and turn on the camera at the same time. When the camera is turned on, start to rotate the platform forward. When the control platform rotates to the origin, start recording the video. When the control platform rotates 360 degrees and returns to the origin, stop recording and complete data collection. The speed control and rotation angle control of the control platform are used to achieve precise control of the start and end of the collected video. No human intervention is required, and fully automatic collection and stitching of images are achieved, which is more accurate and efficient.

[0110] Compared with the prior art, this embodiment can efficiently and accurately stitch cylindrical targets without the need for more computing units, thereby reducing hardware costs. It can also achieve better stitching effects for small targets and images with fewer feature points in two images. The use of an automatically controlled rotating platform can fully realize a fully automatic process without excessive human intervention, thereby improving calculation accuracy and reducing problems such as poor stitching effects caused by misoperation. A fully automatic process is realized without human intervention, greatly reducing the problem of errors caused by human operation.

[0111] For rectangular objects, the existing stitching algorithm mainly segments the image, segments the position of the object and extracts it. The segmentation of the image requires the collection of training models and samples. When the object is replaced, the samples and models need to be updated, which requires a lot of time and manpower. The computational complexity of the segmentation algorithm is relatively large and requires a powerful computing unit to cooperate, which increases too much hardware cost. The embodiment uses background separation for rectangular objects, which has high computational efficiency.

[0112] (1) First, data collection of the background image is performed. After the installation is completed and the position and angle of the camera are debugged, the background image collection is started to obtain the background images RIb1(i,j), RIb2(i,j), RIb3(i,j), RIb4(i,j), RIb5(i,j), RIb1(i,j), RIb2(i,j), RIb3(i,j), RIb4(i,j), RIb5(i,j), which represent five different background images respectively. The pixel values ​​of the background image of the surface, where RIb1(i,j), RIb2(i,j), RIb3(i,j), RIb4(i,j), RIb5(i,j), the subscripts 1, 2, 3, 4, 5 of RIb1(i,j), RIb2(i,j), RIb3(i,j), RIb4(i,j), RIb5(i,j) represent the background of five different surfaces. The five surfaces of the cuboid correspond to five background images, and the surface data of the bottom is not collected. i and j represent the coordinate points of the image.

[0113] (2) After the background image is acquired, the image data of the rectangular target object is acquired. The image acquisition of the rectangular target object mainly involves the acquisition of picture data, starting from the origin of the control platform and starting after rotating 90 degrees. Similarly, one picture is acquired every 90 degrees of rotation. After rotating twice, the top image is acquired. The collected moral image data is RV1(i,j), RV2(i,j), RV3(i,j), RV4(i,j), and RV5(i,j). Where RV1(i,j), RV2(i,j), RV3(i,j), RV4(i,j), RV5(i,j) represent the data of five images respectively. The subscripts 1, 2, 3, 4, 5 of RV1(i,j), RV2(i,j), RV3(i,j), RV4(i,j), RV5(i,j) represent the corresponding subscripts of five background images of different sides. i and j represent the pixel values ​​of the coordinate points of the f frame image at the i and j positions. It is necessary to calculate the position of each current target object and obtain the frame difference result data, so as to calculate the frame difference data between the target image data and the background image data, that is, RF Diffe1(i,j)=RV1(i,j)-RIb1(i,j), RFDiffe2(i,j)=RV2(i,j)-RIb2(i,j), RFDiffe3(i,j)=RV3(i,j)-RIb3(i,j), RFDiffe4(i,j)=RV4(i,j)-RIb4(i,j), RFDiffe5(i,j)=RV5(i,j)-RIb5(i,j). When the frame difference data is greater than the threshold Thre, it means that there is a difference between the target image and the background image. The extracted difference position is the position of the measured rectangular target object.

[0114] (3) The position of the rectangular target object can be calculated based on the above frame difference data. The current position of the rectangular target object, that is, the position of the pixel point, needs to obtain the data of the external rectangle to calculate the position of the pixel point. In this embodiment, the histogram is obtained by means of a histogram. The horizontal histogram and the vertical histogram are first obtained respectively, where:

[0115] The horizontal histogram is RHisti1(i)+=(RFDiffe1(i,j)>Thre?0,1), RHisti2(i)+=(RFDiffe2(i,j)>Thre?0,1), RHisti3(i)+=(RFDiffe3(i,j)>Thre?0,1), RHisti4(i)+=(RFDiffe4(i,j)>Thre?0,1), RHisti5(i)+=(RFDiffe5(i,j)>Thre?0,1),

[0116] The histogram in the vertical direction is RHistj1(j)+=(RFDiffe1(i,j)>Thre?0,1), RHistj2(j)+=(RFDiffe2(i,j)>Thre?0,1), RHistj3(j)+=(RFDiffe3(i,j)>Thre?0,1), RHistj4(j)+=(RFDiffe4(i,j)>Thre?0,1), RHistj5(j)+=(RFDiffe5(i,j)>Thre?0,1).

[0117] Among them, the thresholds in the histogram are Thre_RHisti (horizontal threshold) and Thre_RHistj (vertical threshold). Based on the horizontal histogram threshold and the vertical histogram threshold, the starting position and the ending position of the target object are determined in the horizontal direction and the vertical direction respectively. For example, all the histograms are traversed to find the starting position start_RHisti1, start_RHisti2, start_RHisti4, start_RHisti5 (horizontal direction), start_RHistj1, start _RHistj2, start_RHistj3, start_RHistj4, start_RHistj5 (vertical direction) and the end position end_RHisti1, end_RHisti2, end_RHisti3, end_RHisti4, end_RHisti5 (horizontal direction), end_RHistj1, end_RHistj2, end_RHistj3, end_RHistj4, end_RHistj5 (vertical direction) greater than the threshold, so as to obtain the position coordinates of the rectangular target in the five images:

[0118] x1=start_RHisti1, y1=start_RHistj1, w1=end_RHisti1-start_RHisti1, h1=end_RHistj1-start_RHistj1;

[0119] x2=start_RHisti2, y2=start_RHistj2, w2=end_RHisti2-start_RHisti2, h2=end_RHistj2-start_RHistj2;

[0120] x3=start_RHisti3, y3=start_RHistj3, w3=end_RHisti3-start_RHisti3, h3=end_RHistj3-start_RHistj3;

[0121] x4=start_RHisti4, y4=start_RHistj4, w1=end_RHisti4-start_RHisti4, h4=end_RHistj4-start_RHistj4;

[0122] x5=start_RHisti5, y5=start_RHistj5, w5=end_RHisti5-start_RHisti5, h5=end_RHistj5-start_RHistj5.

[0123] According to the above calculation, the length (unit: pixel number) RL and the height (unit: pixel number) RH of the rectangular target object in the image can be obtained. According to the current data, the effective area in the image can be extracted. The position coordinates of the effective area are RYx1=x1, RYy1=y1, RYw1=w1 and RYh1=h1, RYx2=x2, RYy2=y2, RYw2=w2 and RYh2=h2, RYx3=x3, RYy3=y3, RYw3=w3 and RYh3=h3, RYx4=x4, RYy4=y4, RYw4=w4 and RYh4=h4, RYx5=x5, RYy5=y5, RYw5=w5 and RYh5=h5.

[0124] (4) Extract the effective area image data from each frame of image according to the above-mentioned coordinate diagram, splice the effective area image data, and finally output the spliced ​​image data.

[0125] Among them, for the rectangular target, the main process of platform control and image data collection is: the control platform rotates 90 degrees in the positive direction each time, and after completing the 90-degree rotation, it sends instructions to the image acquisition and starts taking pictures. This process can be completed automatically without human intervention, with higher efficiency and lower error rate. The angle control of each rotation is more precise, which can ensure a more accurate and efficient stitching effect.

[0126] Compared with the prior art, this embodiment can efficiently and accurately stitch rectangular targets without more computing units to complete image segmentation, thereby reducing hardware costs. The use of an automatically controlled rotating platform can fully realize a fully automatic process, and start collecting images after each 90-degree rotation, realizing an automatic collection process without the need for manual triggering of photography, greatly improving the efficiency of use, without excessive human intervention, and can improve the accuracy of calculations, reduce problems such as poor stitching effects caused by misoperation, and realize a fully automatic process without human intervention, greatly reducing the problem of errors caused by human operation.

[0127] In summary, the image processing method provided in this embodiment collects background image data and target image data; calculates frame difference data between the target image data and the background image data; extracts the position information of the target based on the frame difference data; determines the coordinate position and size of the target based on the position information; extracts the effective area containing the target from the target image data based on the coordinate position and size, and generates effective area image data; splices the effective area image data to obtain spliced ​​image data. The image splicing solution provided in this embodiment can accurately determine the position and size of the target through precise frame difference calculation and histogram analysis, thereby improving the accuracy of subsequent image splicing; through automated image acquisition and processing procedures, it reduces manual intervention, improves the efficiency of image splicing, and reduces dependence on computing resources, thereby realizing automated image acquisition and splicing.

[0128] In order to better implement an image processing method of an embodiment of the present application, an image stitching device is also provided in the embodiment of the present application, wherein the meanings of the terms are the same as those in the above-mentioned image processing method, and the specific implementation details can refer to the description in the method embodiment.

[0129] See also Figure 3 , Figure 3 The schematic diagram of the structure of the image stitching device provided in the embodiment of the present application, wherein the image stitching device may specifically include a data acquisition module 201, a frame difference calculation module 202, a position extraction module 203, a target object determination module 204, a region extraction module 205 and an image stitching module 206, which may be specifically as follows:

[0130] The data acquisition module 201 is used to acquire background image data and target image data;

[0131] The frame difference calculation module 202 is used to calculate the frame difference data between the target image data and the background image data;

[0132] A position extraction module 203 is used to extract the position information of the target object based on the frame difference data;

[0133] A target object determination module 204, configured to determine the coordinate position and size of the target object based on the position information;

[0134] The region extraction module 205 is used to extract the effective region containing the target object from the target object image data based on the coordinate position and size, and generate effective region image data;

[0135] The stitching module 206 is used to stitch the image data in the effective area to obtain stitched image data.

[0136] Furthermore, in some embodiments, the data acquisition module 201 is also used to respond to the acquisition instruction and generate an image acquisition instruction for the control platform;

[0137] Based on the image acquisition instruction, the control platform is controlled to rotate according to a preset rotation rule, so as to acquire background image data and target object image data through the camera device on the control platform.

[0138] Specifically, first, according to the preset conditions or the input of the operator, a specific image acquisition instruction is generated to guide the control platform and the camera device to perform subsequent image acquisition work. Through the automated control system, the image acquisition instruction can automatically trigger the camera device to start working without manual intervention, thereby improving the convenience and efficiency of the operation. The control platform rotates according to the image acquisition instruction according to a predetermined rotation rule (for example, uniform rotation or rotation at a specific angle interval) to ensure that the image of the target object is acquired from different angles. The rotation of the platform can be precisely synchronized with the shooting of the camera device to ensure that the image of the target object can be accurately captured at each key angle during the rotation process. In the process of controlling the rotation of the platform, the camera device acquires image data of the background and the target object at a specific angle or continuously according to the acquisition instruction. The acquired data needs to be synchronized with the rotation angle and position of the platform so that the viewing angle corresponding to each image segment can be accurately identified during subsequent processing.

[0139] Furthermore, in some embodiments, the data acquisition module 201 is specifically used to: pre-install and debug the camera device to a preset position and a preset angle; collect background image data of the cylindrical target object through the camera device; collect video image data corresponding to the cylindrical target object starting from the origin of the control platform through the camera device until it rotates 360° to obtain corresponding target object image data.

[0140] Furthermore, in some embodiments, the data acquisition module 201 is specifically used to: pre-install and debug the camera device to a preset position and a preset angle; collect background image data of several different surfaces corresponding to the rectangular target object through the camera device; collect a target object image every 90 degrees from the origin of the control platform through the camera device to obtain corresponding target object image data.

[0141] Furthermore, in some embodiments, the target object determination module 204 may specifically include:

[0142] A histogram unit, used to calculate the histogram corresponding to the target object in the horizontal direction and the vertical direction based on the position information;

[0143] A position unit, used to determine the start position and the end position of the target object in the histogram in the horizontal direction and the vertical direction respectively based on a preset histogram threshold;

[0144] The size unit is used to calculate the coordinate position and size of the target object based on the start position and the end position.

[0145] Furthermore, in some embodiments, the region extraction module 205 may specifically include:

[0146] An area determination unit, used to determine the coordinates corresponding to the effective area containing the target object based on the coordinate position and size of the target object;

[0147] The area extraction unit is used to extract corresponding effective area image data from the target image data based on the coordinates corresponding to the effective area.

[0148] Furthermore, in some embodiments, the image stitching module 206 may specifically include:

[0149] An alignment unit, used for performing image alignment processing on valid image data;

[0150] The stitching unit is used to stitch the effective image data after the alignment processing by using an image fusion algorithm to obtain stitched image data.

[0151] In summary, the image stitching device provided in this embodiment collects background image data and target image data through the data acquisition module 201; the frame difference calculation module 202 is used to calculate the frame difference data between the target image data and the background image data; the position extraction module 203 extracts the position information of the target based on the frame difference data; the target determination module 204 determines the coordinate position and size of the target based on the position information; the region extraction module 205 extracts the effective region containing the target from the target image data based on the coordinate position and size to generate effective region image data; the effective region image data is stitched through the stitching module 206 to obtain the stitched image data. The image stitching device provided in this embodiment can accurately determine the position and size of the target through precise frame difference calculation and histogram analysis, thereby improving the accuracy of subsequent image stitching; through the automated image acquisition and processing process, it reduces manual intervention, improves the efficiency of image stitching, and reduces the dependence on computing resources, thereby realizing automated image acquisition and stitching.

[0152] In addition, the present application also provides an electronic device, such as Figure 4 , which shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present application. Specifically, the electronic device may include a processor 301 with one or more processing cores, a memory 302 with one or more computer-readable storage media, a power supply 303, an input unit 304 and other components. Those skilled in the art can understand that Figure 4The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0153] The processor 301 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 302, and calling data stored in the memory 302, it executes various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 301.

[0154] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and image processing methods by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0155] The electronic device also includes a power supply 303 for supplying power to each component. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions. The power supply 303 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0156] The electronic device may further include an input unit 304, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0157] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 301 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 302 according to the following instructions, and the processor 301 will run the application programs stored in the memory 302, thereby realizing various functions, as follows:

[0158] Collect background image data and target image data; calculate frame difference data between the target image data and the background image data; extract the position information of the target based on the frame difference data; determine the coordinate position and size of the target based on the position information; extract the effective area containing the target from the target image data based on the coordinate position and size, and generate effective area image data; splice the effective area image data to obtain spliced ​​image data.

[0159] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0160] The embodiments of the present application can accurately determine the position and size of the target object through precise frame difference calculation and histogram analysis, thereby improving the accuracy of subsequent image stitching; through automated image acquisition and processing procedures, human intervention is reduced, the efficiency of image stitching is improved, and at the same time, the dependence on computing resources is reduced, thereby realizing automated image acquisition and stitching.

[0161] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0162] To this end, an embodiment of the present application provides a storage medium in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the image processing methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0163] Collect background image data and target image data; calculate frame difference data between the target image data and the background image data; extract the position information of the target based on the frame difference data; determine the coordinate position and size of the target based on the position information; extract the effective area containing the target from the target image data based on the coordinate position and size, and generate effective area image data; splice the effective area image data to obtain spliced ​​image data.

[0164] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0165] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. Since the instructions stored in the storage medium can execute the steps of any image processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any image processing method provided in the embodiments of the present application can be achieved. For details, please refer to the previous embodiments and will not be repeated here.

[0166] The above is a detailed introduction to an image processing method, device, electronic device and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: The steps include: Collecting background image data and target image data; Calculating frame difference data between the target image data and the background image data; Extracting the position information of the target object based on the frame difference data; Determine the coordinate position and size of the target object based on the position information; extracting a valid area including the target object from the target object image data based on the coordinate position and the size, and generating valid area image data; The effective area image data are spliced ​​to obtain spliced ​​image data.

2. The image processing method according to claim 1, characterized in that: The collecting of background image data and target object image data includes: Collecting background image data of the target object by a camera device; The image data corresponding to the target object is collected by the camera device starting from the origin of the control platform until it rotates 360 degrees to obtain the corresponding image data of the target object.

3. The image processing method according to claim 1, characterized in that: The collecting of background image data and target object image data includes: Collecting background image data of several different surfaces corresponding to the target object by a camera device; The camera device collects the target object image every 90 degrees from the origin of the control platform to obtain the corresponding target object image data.

4. The image processing method according to claim 1, characterized in that: The determining the coordinate position and size of the target object based on the position information includes: Calculate the histograms corresponding to the target object in the horizontal direction and the vertical direction based on the position information; Based on a preset histogram threshold, determining the starting position and the ending position of the target object in the histograms in the horizontal direction and the vertical direction respectively; Based on the starting position and the ending position, the coordinate position and the size of the target object are calculated.

5. The image processing method according to claim 1, characterized in that: The step of extracting a valid area including the target object from the target object image data based on the coordinate position and the size to generate valid area image data comprises: Based on the coordinate position and size of the target object, determine the coordinates corresponding to the effective area containing the target object; Based on the coordinates corresponding to the effective area, corresponding effective area image data is extracted from the target image data.

6. The image processing method according to claim 1, characterized in that: The collecting of background image data and target object image data also includes: The control platform rotates according to a preset rotation rule so as to collect background image data and target object image data through the camera device on the control platform.

7. An image processing device, characterized in that: include: A data acquisition module, used for acquiring background image data and target image data; A frame difference calculation module, used for calculating frame difference data between the target image data and the background image data; A position extraction module, used to extract the position information of the target object based on the frame difference data; A target object determination module, used to determine the coordinate position and size of the target object based on the position information; A region extraction module, configured to extract a valid region including the target object from the target object image data based on the coordinate position and the size, and generate valid region image data; The image stitching module is used to stitch the image data in the effective area to obtain stitched image data.

8. The image processing device according to claim 7, characterized in that: The data acquisition module is also used to respond to the acquisition instruction and generate an image acquisition instruction for the control platform; Based on the image acquisition instruction, the control platform is controlled to rotate according to a preset rotation rule, so as to acquire background image data and target object image data through the camera device on the control platform.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the image processing method according to any one of claims 1 to 6 are implemented.

10. A storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the image processing method according to any one of claims 1 to 6.

Images