Image processing method and device, medium, equipment and product

Abstract

The invention relates to an image processing method and device, a medium, equipment and a product, and the method comprises the steps: obtaining a to-be-processed candidate image which is an image containing a tower drum object; determining image change information of the candidate image, wherein the image change information represents image change of the candidate image and an adjacent candidate image of an adjacent frame; and according to the image change information, performing image rotation and / or image translation on the candidate image to obtain a target image.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular, to an image processing method, apparatus, medium, equipment and product. Background Art

[0002] The tower is the pole of wind power generation, which mainly plays a supporting role in wind turbines. Due to the influence of human or environmental factors such as variable load, salt spray erosion, installation defects, etc., the tower is prone to cracks, deformation or flange loosening, causing the tower to deviate from the healthy operating state, which seriously threatens the safe operation of wind turbines. Therefore, in order to ensure the long-term safe and normal operation of wind turbines, it is necessary to measure the vibration of the tower.

[0003] In the prior art, computer vision is usually used to collect tower images for vibration testing. However, in actual applications, due to the influence of external environments such as vehicle driving or strong winds when collecting images, unsatisfactory video files are obtained, making it difficult to obtain accurate vibration signals, resulting in inaccurate vibration measurements. Summary of the invention

[0004] In order to solve the above problems, the present disclosure provides an image processing method, apparatus, medium, device and product.

[0005] In a first aspect, the present disclosure provides an image processing method, the method comprising: Acquire a candidate image to be processed, where the candidate image is an image containing a tower object; Determining image change information of the candidate image, wherein the image change information represents an image change between the candidate image and an adjacent candidate image of an adjacent frame; According to the image change information, the candidate image is rotated and / or translated to obtain a target image.

[0006] Optionally, determining the image change information of the candidate image includes: Acquire adjacent candidate images of adjacent frames of the candidate image; Determining first edge structure information of the candidate image, and determining second edge structure information of the adjacent candidate image; Image change information of the candidate image is determined according to the first edge structure information and the second edge structure information.

[0007] Optionally, determining the first edge structure information of the candidate image includes: Determine a first candidate region in the candidate image, the first candidate region including a stationary reference object; The first candidate region is used as an input of a pre-generated edge detection model to obtain first edge structure information in the first candidate region, where the first edge structure information includes a first edge structure region of the reference object.

[0008] Optionally, determining the second edge structure information of the adjacent candidate image includes: determining a second candidate region in the adjacent candidate image, wherein the second candidate region includes the reference object; The second candidate region is used as an input of the pre-generated edge detection model to obtain second edge structure information in the second candidate region, where the second edge structure information includes the second edge structure region of the reference object.

[0009] Optionally, determining the image change information of the candidate image according to the first edge structure information and the second edge structure information includes: determining, according to the first edge structure information and the second edge structure information, rotation information and / or translation information of the candidate image relative to the adjacent candidate image; The image change information of the candidate image is determined according to the rotation information and / or the translation information.

[0010] Optionally, performing image reconstruction processing on the candidate image according to the image change information includes: When the image change information includes rotation information, the candidate image is rotated according to the rotation information; or, when the image change information includes translation information, the candidate image is translated according to the translation information; or, when the image change information includes rotation information and translation information, the candidate image is rotated and translated according to the rotation information and the translation information; and the target image is obtained.

[0011] Optionally, the edge detection model performs edge detection by the following steps: Obtaining feature maps of multiple preset scales in a sample image; Inputting the multiple feature maps into a pre-trained spatial attention model to obtain a fused feature map, wherein the pixel information of each pixel point on the fused feature map is fused pixel information of multiple preset scales; Determine probability information and threshold information corresponding to the feature map according to the fused feature map, wherein the probability information is used to characterize the probability that the feature map is an edge structure, and the threshold information is used to characterize the probability threshold that the feature map is an edge structure; An edge structure region of the sample image is determined according to the probability information and the threshold information.

[0012] In a second aspect, the present disclosure provides an image processing device, the device comprising: An acquisition module, used for acquiring a candidate image to be processed, wherein the candidate image is an image containing a tower object; A determination module, configured to determine image change information of the candidate image, wherein the image change information represents an image change between the candidate image and an adjacent candidate image of an adjacent frame; A processing module is used to perform image rotation and / or image translation on the candidate image according to the image change information to obtain a target image.

[0013] Optionally, the determining module includes: An acquisition submodule, used for acquiring adjacent candidate images of adjacent frames of the candidate image; A first determining submodule, used to determine first edge structure information of the candidate image, and to determine second edge structure information of the adjacent candidate image; The second determining submodule is used to determine the image change information of the candidate image according to the first edge structure information and the second edge structure information.

[0014] Optionally, the first determination submodule is used to determine a first candidate area in the candidate image, the first candidate area including a stationary reference object; using the first candidate area as input to a pre-generated edge detection model to obtain first edge structure information in the first candidate area, the first edge structure information including a first edge structure area of ​​the reference object.

[0015] Optionally, the first determination submodule is used to determine a second candidate area in the adjacent candidate image, the second candidate area including the reference object; using the second candidate area as input of the pre-generated edge detection model to obtain second edge structure information in the second candidate area, the second edge structure information including the second edge structure area of ​​the reference object.

[0016] Optionally, the second determination submodule is used to determine the rotation information and / or translation information of the candidate image relative to the adjacent candidate image based on the first edge structure information and the second edge structure information; and determine the image change information of the candidate image based on the rotation information and / or the translation information.

[0017] Optionally, the processing module is used to, when the image change information includes rotation information, rotate the candidate image according to the rotation information; or, when the image change information includes translation information, translate the candidate image according to the translation information; or, when the image change information includes rotation information and translation information, rotate and translate the candidate image according to the rotation information and the translation information, to obtain the target image.

[0018] Optionally, the edge detection model performs edge detection by the following steps: Obtaining feature maps of multiple preset scales in a sample image; Inputting the multiple feature maps into a pre-trained spatial attention model to obtain a fused feature map, wherein the pixel information of each pixel point on the fused feature map is fused pixel information of multiple preset scales; Determine probability information and threshold information corresponding to the feature map according to the fused feature map, wherein the probability information is used to characterize the probability that the feature map is an edge structure, and the threshold information is used to characterize the probability threshold that the feature map is an edge structure; An edge structure region of the sample image is determined according to the probability information and the threshold information.

[0019] In a third aspect, the present disclosure provides a non-temporary computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect of the present disclosure.

[0020] In a fourth aspect, the present disclosure provides an electronic device, including: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of the method described in the first aspect of the present disclosure.

[0021] In a fifth aspect, the present disclosure provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect of the present disclosure.

[0022] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects: By acquiring a candidate image to be processed, the candidate image is an image containing a tower object; determining image change information of the candidate image, the image change information characterizing the image change between the candidate image and an adjacent candidate image of an adjacent frame; and performing image rotation and / or image translation on the candidate image according to the image change information to obtain a target image. In this way, the error vibration of the candidate image can be eliminated according to the error vibration caused by the image change of the determined candidate image compared with the adjacent candidate image of the adjacent frame, and the obtained target image can be prevented from being affected by the error vibration, ensuring that the target image only shows the vibration information of the tower itself, thereby improving the accuracy of vibration measurement.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0025] Figure 2 is a structural diagram of an edge detection model according to an exemplary embodiment.

[0026] Figure 3 is a structural diagram of a decoding model according to an exemplary embodiment.

[0027] Figure 4 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0028] Figure 5 is based on Figure 4 The illustrated embodiment shows a block diagram of a determination module.

[0029] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0030] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0031] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0032] The embodiments described in some embodiments of the present disclosure below do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0033] Before introducing the specific implementation methods of the present disclosure in detail, the application scenarios of the present disclosure are first described. The present disclosure can be applied to the vibration measurement scenario of the tower of wind power generation. The tower is the tower pole of wind power generation, which mainly plays a supporting role in the wind turbine generator set. Due to the long-term influence of human or environmental factors such as variable load, salt spray erosion, installation defects, etc., the tower is prone to cracks, deformation or flange loosening, causing the tower to deviate from the healthy operating state, which seriously threatens the safe operation of the wind turbine generator set. Therefore, in order to ensure the long-term safe and normal operation of the wind turbine generator set, it is necessary to measure the vibration of the tower so that maintenance can be carried out in a planned manner according to the actual situation.

[0034] In related technologies, computer vision is usually used to capture tower images for vibration testing. However, in actual applications, due to the influence of external environments such as vehicle movement or strong winds when collecting images, unsatisfactory video files are obtained, making it difficult to obtain accurate vibration signals, resulting in inaccurate vibration measurements.

[0035] In order to overcome the technical problems existing in the above related technologies, the present disclosure provides an image processing method, device, medium, equipment and product. By obtaining a candidate image to be processed, the candidate image is an image containing a tower object; determining the image change information of the candidate image, the image change information characterizes the image change between the candidate image and the adjacent candidate image of the adjacent frame; according to the image change information, the candidate image is rotated and / or translated to obtain a target image. In this way, according to the error vibration caused by the image change of the determined candidate image compared with the adjacent candidate image of the adjacent frame, the error vibration of the candidate image can be eliminated, and the obtained target image can be avoided from being affected by the error vibration, ensuring that the target image only shows the vibration information of the tower itself, and improving the accuracy of vibration measurement.

[0036] The present disclosure is described below in conjunction with specific embodiments.

[0037] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment. Figure 1As shown, the method can be applied to a control device, which can be a common computer, a server, etc., or a vehicle-mounted control device such as a vehicle-mounted computer or a vehicle-mounted industrial personal computer (IPC), or a vehicle controller. The method can include the following steps.

[0038] In step S11, a candidate image to be processed is obtained.

[0039] The candidate image is an image containing a tower object.

[0040] In this step, a high-speed camera may be used to shoot the tower object to obtain a video file, and then multiple image frames may be obtained from the video file, and the multiple image frames may be used as candidate images to be processed. In another possible implementation, the high-speed camera may be used to shoot photos of the tower object according to a preset shooting cycle to obtain multiple photos, and the multiple photos may be used as candidate images to be processed.

[0041] In step S12, image change information of the candidate image is determined.

[0042] The image change information represents the image change between the candidate image and an adjacent candidate image of an adjacent frame.

[0043] In this step, when multiple frames of images are acquired, any frame of the images can be used as the candidate image, and then the adjacent images of the candidate image can be determined from the multiple frames of images, wherein the adjacent image of the candidate image can be the previous frame of image adjacent to the candidate image, or can also be the next frame of image adjacent to the candidate image, and the present disclosure does not limit this. Then, based on the adjacent image and the candidate image, the image change information of the candidate image compared to the adjacent image can be determined, wherein the image change information can be used to characterize the image change between the candidate image and the adjacent candidate image of the adjacent frame.

[0044] Optionally, when the adjacent image is determined, a stationary reference object in the candidate image is first determined.

[0045] Specifically, the candidate image can be identified by an existing target recognition algorithm to determine multiple target objects in the candidate image, and stationary reference objects that are usually stationary, such as street lights, houses, etc., can be determined from the multiple target objects.

[0046] Then, the corresponding stationary reference object in the candidate image can be determined, and then the image changes of the stationary reference object in the image can be determined based on the image area of ​​the stationary reference object in the adjacent image and the image area of ​​the stationary reference object in the candidate image. In this way, since the stationary reference object is in a stationary state, in the two adjacent frames of images, the corresponding image areas do not change when there is no error vibration, but if there is error vibration, the two areas will change. Therefore, the existing error vibration can be reflected by determining the image changes in the two areas.

[0047] In some embodiments, an adjacent candidate image of an adjacent frame of the candidate image may be first acquired; then the first edge structure information of the candidate image may be determined, and the second edge structure information of the adjacent candidate image may be determined; and then the image change information of the candidate image may be determined based on the first edge structure information and the second edge structure information.

[0048] Optionally, a first candidate region in the candidate image may be determined first.

[0049] The first candidate area includes a stationary reference object.

[0050] Specifically, the candidate image can be identified by an existing target recognition algorithm to determine a first candidate area in the candidate image. The first candidate area includes image areas corresponding to stationary reference objects in a stationary state among the identified multiple target objects, for example, image areas corresponding to stationary target objects such as street lamps and houses.

[0051] Then, the first candidate region may be used as an input of a pre-generated edge detection model to obtain first edge structure information in the first candidate region.

[0052] The first edge structure information includes a first edge structure region of the reference object.

[0053] Optionally, the edge detection model performs edge detection through the following steps.

[0054] S1. Obtain feature maps of multiple preset scales in a sample image.

[0055] S2. The multiple feature maps are input into a pre-trained spatial attention model to obtain a fused feature map, where the pixel information of each pixel point on the fused feature map is fused pixel information of multiple preset scales.

[0056] S3. Determine probability information and threshold information corresponding to the feature map according to the fused feature map, wherein the probability information is used to characterize the probability that the feature map is an edge structure, and the threshold information is used to characterize the probability threshold that the feature map is an edge structure.

[0057] S4. Determine the edge structure region of the sample image according to the probability information and the threshold information.

[0058] It should be noted that when the edge detection model is used to perform edge detection on the first candidate area, the sample image is the image corresponding to the first candidate area; and when the edge detection model is used to perform edge detection on the second candidate area, the sample image is the image corresponding to the second candidate area.

[0059] In some embodiments, Figure 2 As shown, the edge detection model includes an encoding model and a decoding model.

[0060] Among them, the encoding model adopts a position-free encoding and hierarchical multi-level encoding design. The encoding model avoids inserting position codes when reasoning on images with different resolutions from the training images, and can easily adapt to arbitrary test image sizes without affecting performance. In addition, the hierarchical multi-level design enables the encoding model to simultaneously generate high-resolution fine features and low-resolution coarse features, which is in contrast to ViT (Vision Transformer, Visual Transformer) which can only generate a single fixed-resolution low-resolution feature map.

[0061] The input image is divided into 4 4 blocks, which is conducive to intensive and fine processing tasks, and these image blocks are used as input to the encoding model. The encoding model is divided into 4 stages, and the features are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image size. The feature block merging process used in ViT is The blocks are unified to This method is used to combine non-overlapping images or features. It cannot maintain the local continuity of the blocks and can use overlapping block merging operations.

[0062] It should be noted that the coding model has good performance, but it also has some limitations: it outputs single-scale low-resolution features instead of multi-scale features; in terms of computational complexity, the computational cost of large-size images is very high. Therefore, an efficient coding model can be designed in the following way to reduce the algorithm complexity while ensuring performance.

[0063] Specifically, the dimension to be reduced can be Reshape the sequence into , and then passed to the fully connected layer, and finally the dimension is sequence.

[0064] And the module parameters of the encoding model are shown in Table 1.

[0065] Table 1 Transformer module parameters

[0066] The decoding model is a lightweight fully convolutional network decoding model that can fuse the encoding model features of multi-stage outputs and output high-quality edge probability maps. The structure of the decoding model is as follows: Figure 3 Usually, the attention of the lower layers in the encoding model tends to stay local, while the attention of the higher layers is highly non-local. Therefore, by aggregating information from different layers, the decoding model can combine local and global information and present a powerful feature representation capability.

[0067] Specifically, the multi-level output features from the encoding model are Convolutional layer processing, the number of output channels in all stages is 128, and stage 1 also adds an output kernel size of The convolutional layer is downsampled to 1 / 2. The outputs of stages 3 and 4 are bilinearly interpolated to the size of stage 2 and concatenated. The decoding model consists of three upsampling blocks, each containing The deconvolution layer and Each module upsamples the input features and performs channel dimension reduction until the original size is reached.

[0068] Then, by calculating the loss function, samples with edge probabilities greater than a preset probability threshold are considered positive samples, and samples with an edge probability of 0 are considered negative samples. The loss of each pixel is calculated to determine the edge structure area of ​​the sample image.

[0069] Optionally, a second candidate region in the adjacent candidate image may be determined first; and the second candidate region is used as an input of the pre-generated edge detection model to obtain second edge structure information in the second candidate region.

[0070] The second candidate region includes the reference object, and the second edge structure information includes a second edge structure region of the reference object.

[0071] It should be noted that the step of determining the second candidate area is consistent with the step of determining the first candidate area mentioned above, and the specific steps will not be repeated here; and the step of outputting the second edge structure information in the second candidate area through the edge detection model is also consistent with the step of deriving the first edge structure information mentioned above, and the specific steps will not be repeated here.

[0072] In one possible implementation, when the first edge structure information and the second edge structure information are obtained, the rotation information and / or translation information of the candidate image relative to the adjacent candidate image can be determined based on the first edge structure information and the second edge structure information; and the image change information of the candidate image can be determined based on the rotation information and / or the translation information.

[0073] In step S13, the candidate image is rotated and / or translated according to the image change information to obtain a target image.

[0074] The image change information may include rotation information and / or translation information.

[0075] Optionally, when the image change information includes rotation information, the candidate image can be rotated according to the rotation information; or, when the image change information includes translation information, the candidate image can be translated according to the translation information; or, when the image change information includes rotation information and translation information, the candidate image can be rotated and translated according to the rotation information and the translation information; then the target image can be obtained after eliminating the error vibration.

[0076] In some embodiments, the image change information may be represented by an image transformation matrix. For example, the target image after eliminating the error vibration may be determined by the following formula: ; ; in, is the image transformation matrix; is the target image; , The candidate images are , Movement of direction; is the rotation angle of the candidate image.

[0077] The above technical solution is adopted to obtain a candidate image to be processed, which is an image containing a tower object; determine the image change information of the candidate image, which represents the image change between the candidate image and the adjacent candidate image of the adjacent frame; and perform image rotation and / or image translation on the candidate image according to the image change information to obtain a target image after eliminating the error vibration. In this way, the error vibration of the candidate image can be eliminated according to the error vibration caused by the image change of the determined candidate image compared with the adjacent candidate image of the adjacent frame, so as to avoid the obtained target image being affected by the error vibration, ensure that the target image only shows the vibration information of the tower itself, and improve the accuracy of vibration measurement.

[0078] Figure 4 is a block diagram of an image processing device according to an exemplary embodiment. Figure 4 The image processing device 200 includes an acquisition module 201, a determination module 202, and a processing module 203.

[0079] An acquisition module 201 is used to acquire a candidate image to be processed, where the candidate image is an image containing a tower object; A determination module 202 is used to determine image change information of the candidate image, where the image change information represents an image change between the candidate image and an adjacent candidate image of an adjacent frame; The processing module 203 is used to perform image rotation and / or image translation on the candidate image according to the image change information to obtain a target image.

[0080] Figure 5 is based on Figure 4 The embodiment shown is a block diagram of a determination module, such as Figure 5 As shown, the determination module 202 includes: An acquisition submodule 2021 is used to acquire adjacent candidate images of adjacent frames of the candidate image; A first determining submodule 2022, configured to determine first edge structure information of the candidate image, and second edge structure information of the adjacent candidate image; The second determining submodule 2023 is configured to determine the image change information of the candidate image according to the first edge structure information and the second edge structure information.

[0081] Optionally, the first determination submodule 2022 is used to determine a first candidate area in the candidate image, and the first candidate area includes a stationary reference object; the first candidate area is used as an input of a pre-generated edge detection model to obtain first edge structure information in the first candidate area, and the first edge structure information includes a first edge structure area of ​​the reference object.

[0082] Optionally, the first determination submodule 2022 is used to determine a second candidate area in the adjacent candidate image, and the second candidate area includes the reference object; the second candidate area is used as an input of the pre-generated edge detection model to obtain second edge structure information in the second candidate area, and the second edge structure information includes the second edge structure area of ​​the reference object.

[0083] Optionally, the second determination submodule 2023 is used to determine the rotation information and / or translation information of the candidate image relative to the adjacent candidate image based on the first edge structure information and the second edge structure information; and determine the image change information of the candidate image based on the rotation information and / or the translation information.

[0084] Optionally, the processing module 203 is used to, when the image change information includes rotation information, rotate the candidate image according to the rotation information; or, when the image change information includes translation information, translate the candidate image according to the translation information; or, when the image change information includes rotation information and translation information, rotate and translate the candidate image according to the rotation information and the translation information, to obtain the target image.

[0085] Optionally, the edge detection model performs edge detection by the following steps: Acquire feature maps of multiple preset scales in a sample image; input the multiple feature maps into a pre-trained spatial attention model to obtain a fused feature map, wherein the pixel information of each pixel point on the fused feature map is fused pixel information of multiple preset scales; determine probability information and threshold information corresponding to the feature map according to the fused feature map, wherein the probability information is used to characterize the probability that the feature map is an edge structure, and the threshold information is used to characterize the probability threshold that the feature map is an edge structure; determine the edge structure region of the sample image according to the probability information and the threshold information.

[0086] The above device is used to obtain a candidate image to be processed, which is an image containing a tower object; determine the image change information of the candidate image, which represents the image change between the candidate image and an adjacent candidate image of an adjacent frame; and perform image rotation and / or image translation on the candidate image according to the image change information to obtain a target image. In this way, the error vibration of the candidate image can be eliminated according to the error vibration caused by the image change of the determined candidate image compared with the adjacent candidate image of the adjacent frame, and the obtained target image can be prevented from being affected by the error vibration, so as to ensure that the target image only shows the vibration information of the tower itself, thereby improving the accuracy of vibration measurement.

[0087] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0088] Figure 6 FIG. 1 is a block diagram of an electronic device 300 according to an exemplary embodiment. Figure 6 As shown, the electronic device 300 may include: a processor 301 , a memory 302 . The electronic device 300 may also include one or more of a multimedia component 303 , an input / output (I / O) interface 304 , and a communication component 305 .

[0089] The processor 301 is used to control the overall operation of the electronic device 300 to complete all or part of the steps in the above-mentioned image processing method. The memory 302 is used to store various types of data to support the operation of the electronic device 300. For example, these data may include instructions for any application or method used to operate on the electronic device 300, and application-related data, such as contact data, messages sent and received, pictures, audio, video, etc. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 303 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 302 or sent through the communication component 305. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 304 provides an interface between the processor 301 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 305 is used for wired or wireless communication between the electronic device 300 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 305 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0090] In an exemplary embodiment, the electronic device 300 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned image processing method.

[0091] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned image processing method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 302 including program instructions, and the above-mentioned program instructions can be executed by the processor 301 of the electronic device 300 to complete the above-mentioned image processing method.

[0092] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-mentioned image processing method when executed by the programmable device.

[0093] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0094] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0095] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a candidate image to be processed, where the candidate image is an image containing a tower object; Determining image change information of the candidate image, wherein the image change information represents an image change between the candidate image and an adjacent candidate image of an adjacent frame; According to the image change information, the candidate image is rotated and / or translated to obtain a target image.

2. The method according to claim 1, characterized in that Determining the image change information of the candidate image includes: Acquire adjacent candidate images of adjacent frames of the candidate image; Determining first edge structure information of the candidate image, and determining second edge structure information of the adjacent candidate image; Image change information of the candidate image is determined according to the first edge structure information and the second edge structure information.

3. The method according to claim 2, characterized in that Determining the first edge structure information of the candidate image includes: Determine a first candidate region in the candidate image, the first candidate region including a stationary reference object; The first candidate region is used as an input of a pre-generated edge detection model to obtain first edge structure information in the first candidate region, where the first edge structure information includes a first edge structure region of the reference object.

4. The method according to claim 3, characterized in that The determining the second edge structure information of the adjacent candidate image comprises: determining a second candidate region in the adjacent candidate image, wherein the second candidate region includes the reference object; The second candidate region is used as an input of the pre-generated edge detection model to obtain second edge structure information in the second candidate region, where the second edge structure information includes the second edge structure region of the reference object.

5. The method according to claim 4, characterized in that The determining the image change information of the candidate image according to the first edge structure information and the second edge structure information comprises: determining, according to the first edge structure information and the second edge structure information, rotation information and / or translation information of the candidate image relative to the adjacent candidate image; The image change information of the candidate image is determined according to the rotation information and / or the translation information.

6. The method according to claim 5, characterized in that The performing image reconstruction processing on the candidate image according to the image change information comprises: In the case where the image change information includes rotation information, rotating the candidate image according to the rotation information; or, In the case where the image change information includes translation information, performing image translation on the candidate image according to the translation information; or, In a case where the image change information includes rotation information and translation information, performing image rotation and image translation on the candidate image according to the rotation information and the translation information; The target image is obtained.

7. The method according to claim 3 or 4, characterized in that: The edge detection model performs edge detection through the following steps: Obtaining feature maps of multiple preset scales in a sample image; Inputting the multiple feature maps into a pre-trained spatial attention model to obtain a fused feature map, wherein the pixel information of each pixel point on the fused feature map is fused pixel information of multiple preset scales; Determine probability information and threshold information corresponding to the feature map according to the fused feature map, wherein the probability information is used to characterize the probability that the feature map is an edge structure, and the threshold information is used to characterize the probability threshold that the feature map is an edge structure; An edge structure region of the sample image is determined according to the probability information and the threshold information.

8. An image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a candidate image to be processed, wherein the candidate image is an image containing a tower object; A determination module, configured to determine image change information of the candidate image, wherein the image change information represents an image change between the candidate image and an adjacent candidate image of an adjacent frame; A processing module is used to perform image rotation and / or image translation on the candidate image according to the image change information to obtain a target image.

9. The device according to claim 8, characterized in that The determination module comprises: An acquisition submodule, used for acquiring adjacent candidate images of adjacent frames of the candidate image; A first determining submodule, used to determine first edge structure information of the candidate image, and to determine second edge structure information of the adjacent candidate image; The second determining submodule is used to determine the image change information of the candidate image according to the first edge structure information and the second edge structure information.

10. The device according to claim 9, characterized in that The first determination submodule is used to determine a first candidate area in the candidate image, wherein the first candidate area includes a stationary reference object; the first candidate area is used as an input of a pre-generated edge detection model to obtain first edge structure information in the first candidate area, wherein the first edge structure information includes a first edge structure area of ​​the reference object.

11. The device according to claim 10, characterized in that The first determination submodule is used to determine a second candidate area in the adjacent candidate image, wherein the second candidate area includes the reference object; and use the second candidate area as an input to the pre-generated edge detection model to obtain second edge structure information in the second candidate area, wherein the second edge structure information includes a second edge structure area of ​​the reference object.

12. The device according to claim 11, characterized in that The second determination submodule is used to determine the rotation information and / or translation information of the candidate image relative to the adjacent candidate image based on the first edge structure information and the second edge structure information; and determine the image change information of the candidate image based on the rotation information and / or the translation information.

13. The device according to claim 12, characterized in that The processing module is used to rotate the candidate image according to the rotation information when the image change information includes rotation information; or, when the image change information includes translation information, translate the candidate image according to the translation information; or, when the image change information includes rotation information and translation information, rotate and translate the candidate image according to the rotation information and the translation information; to obtain the target image.

14. The device according to claim 10 or 11, characterized in that The edge detection model performs edge detection through the following steps: Obtaining feature maps of multiple preset scales in a sample image; Inputting the multiple feature maps into a pre-trained spatial attention model to obtain a fused feature map, wherein the pixel information of each pixel point on the fused feature map is fused pixel information of multiple preset scales; Determine probability information and threshold information corresponding to the feature map according to the fused feature map, wherein the probability information is used to characterize the probability that the feature map is an edge structure, and the threshold information is used to characterize the probability threshold that the feature map is an edge structure; An edge structure region of the sample image is determined according to the probability information and the threshold information.

15. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

16. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images