Picture direction recognition method and device, equipment, storage medium and program product

Abstract

The invention discloses a picture direction recognition method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring a to-be-recognized picture; taking a preset reference as a benchmark, respectively rotating the to-be-identified picture at n-1 preset angles to obtain n-1 reference pictures in different rotation directions, n being a positive integer greater than or equal to 2; image convolution operation is carried out on each target picture, a corresponding group of image features is extracted, and the target pictures comprise n-1 reference pictures and pictures to be recognized; respectively inputting each group of image features into a pre-trained object recognition model to obtain a recognition result, the recognition result comprising an object existing in the target picture and position information of the object; and according to each identification result, determining the actual direction of the to-be-identified picture in the n identification results. According to the embodiment of the invention, the accuracy of picture direction recognition can be effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and in particular relates to an image direction recognition method, device, equipment, storage medium and program product. Background Art

[0002] Artificial intelligence image intelligent recognition technology is increasingly used in network management, IT (Information Technology) system monitoring and other fields. These technologies require intelligent recognition of images such as daily operation and maintenance inspections and user experience abnormal photos, so as to detect abnormal situations in a timely manner. However, the 90°, 180°, and 270° rotations that may occur during the shooting or transmission of images may make it difficult for the intelligent recognition system to accurately identify the image content, thereby affecting monitoring efficiency.

[0003] To accurately identify the orientation of an image, a common strategy is to use the Exchangeable Image File Format (EXIF) information of the image to record and determine the correct orientation of the image. EXIF ​​information is a standard for storing image metadata, including shooting information, camera settings, and image orientation. Most smartphones and some cameras can record the orientation of the photo when taking the photo through the built-in gyroscope, providing orientation information for the display and processing of the image.

[0004] However, the accuracy of EXIF ​​information is limited by many factors. For example, when a mobile phone is placed horizontally and shoots a document from top to bottom, the built-in gyroscope may not accurately capture the actual shooting direction, resulting in deviations in the recorded direction information, which may affect the accuracy of the image direction recognition. Summary of the invention

[0005] The embodiments of the present invention provide a method, device, equipment, storage medium and program product for identifying the direction of an image, which can effectively improve the accuracy of identifying the direction of an image.

[0006] In a first aspect, an embodiment of the present application provides a method for identifying the direction of an image, the method comprising:

[0007] Get the image to be identified;

[0008] Based on the preset reference, the image to be identified is rotated by n-1 preset angles to obtain n-1 reference images with different rotation directions, where n is a positive integer greater than or equal to 2;

[0009] Perform image convolution operations on each target image to extract a corresponding set of image features. The target image includes n-1 reference images and the image to be identified.

[0010] Input each set of image features into a pre-trained object recognition model to obtain a recognition result, which includes the objects existing in the target image and the location information of the objects;

[0011] According to each recognition result, the actual direction of the image to be recognized is determined among the n recognition results.

[0012] In a second aspect, an embodiment of the present application provides a device for identifying the direction of an image, the device comprising:

[0013] A first acquisition module is used to acquire a picture to be identified;

[0014] The first rotation module is used to rotate the image to be identified by n-1 preset angles based on a preset reference, to obtain n-1 reference images with different rotation directions, where n is a positive integer greater than or equal to 2;

[0015] The first extraction module is used to perform image convolution operation on each target image to extract a corresponding set of image features, where the target image includes n-1 reference images and the image to be identified;

[0016] A first determination module is used to input each set of image features into a pre-trained object recognition model to obtain a recognition result, where the recognition result includes objects existing in the target image and location information of the objects;

[0017] The second determination module is used to determine the actual direction of the image to be identified among the n identification results according to each identification result.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;

[0019] When the processor executes the computer program instructions, it implements the image direction recognition method as described in any one of the first aspects.

[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, a method for identifying the direction of an image as described in any one of the first aspects is implemented.

[0021] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program, and when the computer program is processed and executed, it implements the image direction recognition method of any one of the first aspects.

[0022] The image direction recognition method, device, equipment, storage medium and program product of the embodiments of the present application can capture the multi-angle image features of the image to be recognized in different rotation directions by acquiring n groups of image features of the image to be recognized. Then, these multi-angle image features are input into a pre-trained object recognition model, and the content learned by the object recognition model can be used to accurately analyze and recognize the objects contained in the image to be recognized in different directions. Furthermore, by analyzing the recognition results of the image to be recognized in multiple directions, it can be evaluated which direction has the highest consistency and accuracy of the recognition results, thereby determining the actual direction of the image to be recognized.

[0023] This method does not rely on the EXIF ​​information of the image or the gyroscope of the device, but judges the direction through the recognition results (objects and their location information), that is, the content of the image. Specifically, by recognizing the results of the image in different rotation directions, the direction of the object in the image can be evaluated from multiple angles, thereby bypassing the direction recognition bias caused by hardware limitations or information loss, and effectively improving the accuracy of the direction recognition of the image to be recognized. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0025] Figure 1 A schematic diagram of a process flow of a method for identifying the direction of an image provided by an embodiment of the present application is shown;

[0026] Figure 2 A flowchart of a specific implementation method of S103 provided in an embodiment of the present application is shown;

[0027] Figure 3 A schematic diagram showing a flow chart of a method for identifying the direction of an image provided by another embodiment of the present application is shown;

[0028] Figure 4 A flowchart of a specific implementation method of S302 provided in an embodiment of the present application is shown;

[0029] Figure 5 A flowchart of a specific implementation method of S3021 provided in an embodiment of the present application is shown;

[0030] Figure 6 A flowchart of a specific implementation method of S105 provided in an embodiment of the present application is shown;

[0031] Figure 7A flowchart of another specific implementation of S105 provided in an embodiment of the present application is shown;

[0032] Figure 8 A schematic diagram showing the structure of a device for identifying the direction of an image provided by an embodiment of the present application is shown;

[0033] Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0035] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0036] In the current field of image processing, a variety of technical methods have been proposed to ensure the correct orientation of the image. The following is an overview of several major methods:

[0037] 1. Use the EXIF ​​information of the picture:

[0038] The gyroscope records the orientation of the photo when it is taken and saves this information in the EXIF ​​information of the photo. Subsequent image display and processing can determine the orientation of the photo by parsing this EXIF ​​information.

[0039] This method of using the EXIF ​​information of an image to identify the image orientation relies on the device being able to accurately record orientation information when taking a photo. However, not all devices support EXIF ​​information, and even if they do, the EXIF ​​information may be inaccurate or lost due to operations such as screenshots and compression of the original image.

[0040] 2. Image augmentation training recognition model:

[0041] By performing augmented training on four directions of the original image (original, rotated 90°, rotated 180°, and rotated 270°), a recognition model that is insensitive to image direction is obtained, so that the obtained recognition model can handle image recognition processing in various directions.

[0042] However, this method requires a lot of computing resources for model training and is only applicable to specific types of image recognition tasks, and cannot handle the direction recognition of general images. For images that only need to show the correct direction, this augmented training may be inefficient.

[0043] 3. Training a special classifier for image orientation:

[0044] By training a large number of sample images in different directions, direct output of four direction classifications can be achieved.

[0045] However, this method requires ensuring that the number of sample images in each direction is relatively balanced, but in reality, most images are positive by default. Secondly, if training is performed by rotating images, the amount of training will increase significantly (usually four times the original amount of training), which not only increases the consumption of computing resources, but may also cause the model performance to deteriorate due to the large features of the entire image.

[0046] In order to solve the above technical problems, the embodiments of the present application provide a method, device, equipment, storage medium and program product for image direction recognition. The following first introduces the image direction recognition method provided by the embodiments of the present application.

[0047] Figure 1 FIG. 1 is a flow chart of a method for identifying the direction of an image provided by an embodiment of the present application. Figure 1 As shown, the image direction recognition method may include the following steps: S101-S105.

[0048] S101, obtaining a picture to be identified.

[0049] S102, based on a preset reference, the image to be identified is rotated by n-1 preset angles to obtain n-1 reference images with different rotation directions, where n is a positive integer greater than or equal to 2.

[0050] S103, performing image convolution operation on each target image to extract a corresponding set of image features, where the target image includes n-1 reference images and the image to be identified.

[0051] S104, input each set of image features into a pre-trained object recognition model to obtain a recognition result, which includes the objects existing in the target image and the location information of the objects.

[0052] S105, according to each recognition result, determining the actual direction of the image to be recognized among the n recognition results.

[0053] In an embodiment of the present application, by acquiring n groups of image features of the image to be identified, the image features of multiple angles in different rotation directions of the image to be identified can be captured. Then, these multi-angle image features are input into a pre-trained object recognition model, and the content learned by the object recognition model can be used to accurately analyze and identify the objects contained in the image to be identified in different directions. Furthermore, by analyzing the recognition results of the image to be identified in multiple directions, it is possible to evaluate which direction has the highest consistency and accuracy of the recognition results, thereby determining the actual direction of the image to be identified.

[0054] This method does not rely on the EXIF ​​information of the image or the gyroscope of the device, but judges the direction through the recognition results (objects and their location information), that is, the content of the image. Specifically, by recognizing the results of the image in different rotation directions, the direction of the object in the image can be evaluated from multiple angles, thereby bypassing the direction recognition bias caused by hardware limitations or information loss, and effectively improving the accuracy of the direction recognition of the image to be recognized.

[0055] In some embodiments, in S102, n-1 preset angles can reflect different postures and directions of the object in the picture to be identified in different rotation directions. In specific implementation, first, the picture to be identified is aligned with a preset reference point or original direction to ensure the baseline consistency of the rotation operation. Then, the picture to be identified is rotated by n-1 preset angles to ensure that the required viewing angle range is covered. Finally, n pictures at different angles are obtained, and the n pictures at different angles include n-1 reference pictures and pictures to be identified. These n pictures at different angles provide views of the picture to be identified at multiple rotation angles, thereby realizing multi-angle analysis of the content of the picture to be identified.

[0056] Specifically, we can choose to rotate 90°, 180°, and 270°, plus the original 0° (or equivalently, 360°), for a total of four angles. These four angles divide the plane into four quadrants, each of which represents a basic direction. For example, 0° represents the original direction, which is usually aligned with the horizontal direction of the image; rotating 90° means that the image is rotated one quadrant clockwise, and the object will be presented in a vertical posture; rotating 180° means that the image is rotated half a circle, and the direction of the object is completely opposite to the original direction; and rotating 270° means that the image is rotated one quadrant clockwise again, and the object will return to a posture that is close to horizontal but in the opposite direction. Through these four preset angles, we can comprehensively cover all possible basic rotation directions of the image, ensuring that the different postures and directions of the object can be accurately identified during the image direction recognition process.

[0057] In some embodiments, in S103, the image convolution operation can extract meaningful local image features in the corresponding target image, such as corners, edges, textures, etc. In the specific operation, a small matrix (convolution kernel or filter) can be slid across the entire image matrix, and the sum of the corresponding multiplication of the convolution kernel and the elements of the local area of ​​the image is calculated at each position.

[0058] In specific implementation, the image convolution operation can be performed on the target images one by one in a preset order (such as clockwise or counterclockwise). By performing the convolution operation on each target image, the local image features of the image can be extracted, which is helpful for subsequent image analysis and processing tasks.

[0059] Figure 2 A flowchart of a specific implementation method of S103 provided in an embodiment of the present application is shown.

[0060] In order to improve the computational efficiency and accuracy of image features. Figure 2 As shown, the aforementioned S103 may specifically include performing the following operations respectively for each target image: S1031 to S1036.

[0061] S1031: Transform the target image using Fourier transform to obtain a frequency domain vector.

[0062] Among them, Fourier transform is used to analyze the frequency content of an image. It is a mathematical operation used to convert the brightness or color value of each pixel of the target image into the corresponding frequency component. In the result of Fourier transform, each pixel corresponds to a value in the frequency domain, and these values ​​​​compose a frequency domain vector.

[0063] S1032: Obtain the convolution kernel corresponding to the target image.

[0064] S1033: Transform the convolution kernel to obtain a transformed convolution kernel.

[0065] In some embodiments, the convolution kernel is transformed using the Winograd algorithm, which is an algorithm for accelerating convolution operations and reduces the amount of calculation by performing specific mathematical transformations on the convolution kernel and input data.

[0066] S1034: Perform a point multiplication on the frequency domain vector and the transformed convolution kernel to obtain an intermediate result.

[0067] Among them, the frequency domain vector is point-multiplied with the transformed convolution kernel to realize frequency domain convolution, that is, to obtain the filtered frequency domain representation, which helps to identify the frequency features in the target image.

[0068] S1035: Perform inverse Fourier transform on the intermediate result to obtain a convolution result.

[0069] Among them, the intermediate result after frequency domain convolution is inverse Fourier transformed to convert the data from frequency domain back to time domain and obtain the convolution result in time domain. These convolution results represent the characteristic response of the image after being processed by the convolution kernel in the time domain.

[0070] S1036: Extract a set of image features representing the target image based on the convolution result.

[0071] In the embodiment of the present application, by combining the fast Fourier transform (FFT) and the Winograd algorithm, image convolution operations are performed on each picture respectively, which can greatly reduce the amount of data while retaining the image features, and effectively improve the efficiency and accuracy of image feature calculation.

[0072] In a specific implementation scenario, first, perform a fast Fourier transform (FFT) on the input vector x corresponding to the target image, convert it from the time domain to the frequency domain, and obtain a frequency domain vector X, X = FFT (x). Then, use the Winograd algorithm to transform the convolution kernel h corresponding to the target image, and obtain the transformed convolution kernel H, H = WinogradTransform (h), WinogradTransform (h) represents the Winograd algorithm transformation of the convolution kernel h. Next, perform a matrix multiplication operation on the frequency domain vector X and the transformed convolution kernel H to obtain the intermediate result Z, Z = X*H T , where X represents the frequency domain vector, H T Represents the transposed matrix of the convolution kernel H. After that, the intermediate result Z is subjected to an inverse fast Fourier transform (IFFT) and converted from the frequency domain back to the time domain to obtain a time domain vector Y, Y = IFFT (Z). Finally, in order to improve the smoothness and stability of the output result, the time domain vector Y is smoothed by a sliding average to obtain the final output result.

[0073] Figure 3 A schematic flow chart of a method for identifying image direction provided in another embodiment of the present application is shown.

[0074] In some embodiments, in order to accurately obtain the recognition result corresponding to each set of image features, such as Figure 3 As shown, before the aforementioned S104, the image direction recognition method further includes the following steps: S301-S302.

[0075] S301, obtaining a training sample set, where the training sample set includes multiple training samples, each training sample includes a picture and a label corresponding to the picture, and the label includes objects included in the picture and location information corresponding to each object.

[0076] In some embodiments, first, image data containing objects of various categories may be collected and organized. Then, a labeling tool may be used to label each image to determine the object category and location information in the image. Next, a training sample set containing images and corresponding labels may be constructed for use in model training.

[0077] S302, using a plurality of training samples to train a pre-built initial recognition model until a training stop condition is met, thereby obtaining a trained object recognition model.

[0078] In the embodiment of the present application, the initial recognition model is trained on a large number of labeled training sample sets, so that the initial recognition model can learn the features that distinguish different objects. The trained object recognition model obtained until a certain training stop condition is met can learn the ability to classify image features, improve the accuracy of object recognition, and the model can automatically identify objects in the picture and their corresponding positions, effectively reducing manual intervention and improving the efficiency of object recognition.

[0079] As an example, a direction-sensitive general object recognition model can be trained. Based on this, existing large datasets can be used as a basis to build a training sample set, such as MS COCO (Microsoft COCO: Common Objects in Context). The MS COCO dataset provides a variety of object categories and detailed annotation information.

[0080] It is worth noting here that when training an object recognition model that is sensitive to the direction of an image, it is necessary to omit the standard rotation augmentation training operation in the augmented part of the training sample set to ensure that the model can focus on the images in the positive direction for object recognition, while remaining insensitive to the rotated images. In this way, the object recognition model's ability to recognize objects at different angles in real scenes can be effectively improved. At the same time, in order to improve the robustness of the object recognition model in practical applications, it is necessary to retain other types of image augmentation processing methods, such as color augmentation, which helps the image recognition model adapt to different lighting conditions and background environments. Color augmentation simulates various lighting changes that may occur in real environments by adjusting the image's brightness, contrast, saturation and other properties, so that the object recognition model can better cope with these changes and more accurately identify the target object.

[0081] In a specific implementation scenario, when the object recognition model is applied to a specific operation and maintenance scenario, the standard data set may not be sufficient to cover all important object categories. Common operation and maintenance objects can be added as content of the training sample set to enhance the pertinence and practicality of the training sample set.

[0082] Specifically include:

[0083] (1) All graph types based on Echarts are labeled and added to the training sample set to ensure that the object recognition model can recognize common charts in the operation and maintenance process, such as line charts, bar charts, or pie charts.

[0084] (2) For Android, Microsoft Windows, macOS, iOS (Internetworking

[0085] The desktop environments of operating systems such as the OS and other operating systems are labeled and added to the training sample set to ensure that the object recognition model can accurately distinguish between different operating system interfaces.

[0086] (3) Integrate multiple labeled text recognition training sets such as Chinese street view text provided by PaddleOCR to enhance the object recognition model's ability to process Chinese text and improve its text recognition accuracy in complex images.

[0087] Through the above enhancement measures, the object recognition model obtained can simultaneously recognize common objects, desktop environments, charts and text in images in operation and maintenance scenarios, providing more comprehensive and accurate information support for operation and maintenance personnel, which helps to improve operation and maintenance efficiency and accuracy.

[0088] The aforementioned initial recognition model can be constructed based on the architecture of a convolutional neural network, and during the training process, a standard convolutional neural network can be used for object recognition training. Commonly used convolutional neural network architectures include ResNet (residual network) and VGG (Visual Geometry Group) architectures with good feature extraction capabilities. ResNet effectively alleviates the gradient vanishing problem in deep networks by introducing residual blocks, allowing the network to be trained deeper and more effectively. VGG improves performance by increasing the depth and width of the network, and has achieved significant results on multiple data sets.

[0089] Furthermore, in order to achieve higher directional sensitivity, a specifically designed rotation-invariant convolution kernel can be used to extract rotation-invariant features in the image. The rotation-invariant convolution kernel has specific weights and bias terms that can capture the directional characteristics of objects in the image, thereby enhancing the model's sensitivity to directional changes.

[0090] By omitting rotation augmentation, using rotation-invariant convolution kernels, and retaining other types of image augmentation processing methods, the object recognition model can be made more sensitive to changes in image orientation and more robust under different lighting and background conditions.

[0091] In a specific implementation scenario, suppose we have an input image that contains an upward arrow. Using a rotation-invariant convolution kernel to convolve this image, we can extract the rotation-invariant features of the arrow. These features will not depend on the direction of the arrow, so the trained object recognition model will be able to recognize arrows rotated to any angle.

[0092] Figure 4 A flowchart of a specific implementation method of S302 provided in an embodiment of the present application is shown.

[0093] In order to accurately identify and obtain the recognition results corresponding to each set of image features, such as Figure 4 As shown, the aforementioned S302 may specifically include the following steps: S3021 to S3025.

[0094] S3021: For each training sample, respectively perform: input the image into the convolution layer of the convolutional neural network, and use the rotation invariant convolution kernel introduced in the convolution layer to extract the feature vector of the image.

[0095] S3022: Perform nonlinear transformation on the feature vector using an activation function to obtain a transformed feature representation.

[0096] S3023: Input the transformed feature representation into the fully connected layer, and use the fully connected layer to aggregate and classify the transformed features to obtain the recognition result of the image.

[0097] In some embodiments, the image is input into the convolution layer of a convolutional neural network (CNN). In the convolution layer, a rotationally invariant convolution kernel is used to extract features of the image. Subsequently, the convolution layer passes the extracted feature vector to an activation function, such as a ReLU (Rectified Linear Unit) function, a Sigmoid function, or a Tanh (hyperbolic tangent) function. The activation function performs a nonlinear transformation on the feature vector to obtain a transformed feature representation. Next, the feature representation transformed by the activation function is further input into a fully connected layer. The fully connected layer is used to aggregate and classify the transformed features to obtain a recognition result.

[0098] S3024: Determine the loss function value of the initial recognition model based on the recognition results of each training sample.

[0099] S3025: When the loss function value satisfies a preset training stop condition, a trained object recognition model is obtained.

[0100] In some embodiments, based on the recognition results obtained after each training sample passes through the initial recognition model, a loss function value is calculated to measure the difference between the model prediction and the true value. Subsequently, we compare this loss function value with the preset training stop condition. When the loss function value meets the preset stop condition (for example, the loss value in multiple consecutive iteration cycles no longer decreases significantly, or reaches the preset maximum number of iterations, etc.), it is determined that the model has completed effective training, and the model at this time is used as the trained object recognition model.

[0101] In the embodiment of the present application, the rotation-invariant convolution kernel in the convolution layer can extract the rotation-invariant features in the image, so that the convolutional neural network can capture the directional features of the objects in the image, thereby enhancing the sensitivity of the object recognition model to directional changes. Moreover, in the case of the same original image training set, compared with multi-directional or full-angle training, the cost of training the model is greatly reduced.

[0102] Figure 5 A flowchart of a specific implementation method of S3021 provided in an embodiment of the present application is shown.

[0103] In order to accurately extract the rotation invariant features in the image, such as Figure 5 As shown, the aforementioned S3021 may specifically include the following steps: S3021a~S3021d.

[0104] S3021a: Calculate the gradient information of the image in the horizontal and vertical directions and construct a gradient map.

[0105] Among them, gradient information can be obtained through edge detection filters such as the Sobel operator and the Prewitt operator. In edge detection tasks, gradients can describe the direction and intensity of changes in pixel brightness in an image, while edges are areas where these changes are obvious. The Sobel operator and the Prewitt operator are both discrete differential operators used to calculate the first-order, second-order, or mixed image derivatives of image brightness functions. These operators are based on convolution operations, which detect brightness changes between pixels by sliding a small window (convolution kernel or filter) across the image.

[0106] Specifically, both the Sobel operator and the Prewitt operator include two convolution kernels, one for detecting edges in the horizontal direction and the other for detecting edges in the vertical direction. These two convolution kernels are usually convolved with the image to generate two new images, representing the gradient strength in the horizontal and vertical directions respectively. Then, the edge strength of each pixel can be obtained by calculating the square root of the sum of the squares of these two gradient strengths.

[0107] S3021b: For each position of the gradient map, the corresponding convolution kernel response is calculated using the rotation invariant convolution kernel.

[0108] As a specific example, a rotationally invariant convolution kernel is expressed as follows:

[0109] w[i,j]=cosθ*x[i:i+f,j:j+f]+sinθ*y[i:i+f,j:j+f]

[0110] Wherein, w[i,j] represents the response value calculated by the rotation invariant convolution kernel at the position (i,j), i and j are the position coordinates in the row and column directions respectively, x and y represent the gradient information of the input image on the x-axis and y-axis respectively, θ represents the preset rotation angle, cosθ and sinθ are used to adjust the contribution of the gradient information of the input image in the x-axis and y-axis respectively, f represents the size of the convolution kernel, x[i:i+f,j:j+f] represents the expansion of the gradient information area of ​​f in the x-axis direction starting from the position (i,j), and y[i:i+f,j:j+f] represents the expansion of the gradient information area of ​​f in the y-axis direction starting from the position (i,j). In the specific implementation, the rotation invariant convolution kernel is used to traverse the entire gradient map. The response value at each position on the gradient map is calculated using the expression of the rotation invariant convolution kernel mentioned above.

[0111] In conjunction with the arrow detection in the aforementioned implementation scenario, this rotation-invariant convolution kernel does not care about the direction the arrow actually points to, but only focuses on the shape features of the arrow. Therefore, no matter how the arrow in the input image to be identified is rotated, as long as its shape features are reflected in the gradient map, this rotation-invariant convolution kernel can identify the existence of the arrow by identifying the gradient pattern that matches the rotation angle.

[0112] S3021c: Construct a feature map of the image using the gradient map and multiple convolution kernel responses, where the feature map includes feature information that is rotationally invariant.

[0113] S3021d: Convert the feature map into a feature vector.

[0114] In the embodiment of the present application, the design of the rotation-invariant convolution kernel enables the object recognition model to extract similar features at different rotation angles, thereby ensuring that the feature map is rotationally invariant. The feature map incorporates the feature information in the image that is rotationally invariant, and this feature information can remain stable when the image is rotated, thereby enhancing the model's robustness to rotation changes.

[0115] In some embodiments, in S104, for the image features of the input object recognition model, first, a multi-scale sliding window is used to generate an initial region proposal. Subsequently, non-maximum suppression (NMS) is used to eliminate redundant detection boxes for these initial region proposals to ensure that each object is detected only once. Next, overlapping region proposals are eliminated using overlapping pooling (RoIPooling) to improve the accuracy and efficiency of detection. After that, each region proposal is assigned to a specific level based on its feature information and location information. Larger region proposals are assigned to feature map levels with lower resolutions, while smaller region proposals are assigned to higher resolution levels. After that, for the region proposals in each level, Faster R-CNN, which includes a region proposal network and a classifier, is used to perform final object classification and position adjustment. In this way, the position and category of objects in the picture can be directly predicted, thereby improving the accuracy and efficiency of detection.

[0116] As an example, the final recognition result can output the classification C and position P information of each object contained in the image in each direction state. Each direction outputs C1~C n and P1~P n , n is the number of common objects recognized.

[0117] Figure 6 A flowchart of a specific implementation method of S105 provided in an embodiment of the present application is shown.

[0118] The number of objects output by the image features corresponding to each direction is not only the same, therefore, in order to determine the actual direction of the image to be identified, such as Figure 6 , the aforementioned S105 may specifically include the following steps: S1051a~S1052a.

[0119] S1051a, searching for the first target recognition result having the largest number of objects among the n recognition results;

[0120] S1052a: Determine the direction of the target image corresponding to the first target recognition result as the actual direction of the image to be recognized.

[0121] In the embodiment of the present application, the accuracy of image direction recognition directly affects the accuracy of object recognition. When the actual direction of the image to be recognized is correctly recognized, the number of objects recognized is the largest, that is, the recognition result with the largest number of recognized objects is the result that best matches the image to be recognized. Therefore, the direction corresponding to the first target recognition result with the largest number of objects can be directly determined as the actual direction of the image to be recognized, which helps to improve the efficiency of image direction recognition.

[0122] Figure 7 A flowchart of another specific implementation method of S105 provided in an embodiment of the present application is shown.

[0123] In order to further improve the accuracy of determining the actual direction of the image to be identified, Figure 7 , the aforementioned S105 may specifically include the following steps: S1051b~S1053b.

[0124] S1051b: For each recognition result, determine a total contribution value of the recognition result according to a rotation weight corresponding to each object in the recognition result, where the rotation weight is determined based on the occurrence rate of rotation actions of the object in the actual scene.

[0125] As an example, the total contribution is a composite value calculated based on the rotation weight of each object in the recognition result, which is used to evaluate the contribution of the recognition result to determining the actual orientation of the image. Buildings are rarely rotated, so the corresponding rotation weight of buildings may be low; while vehicles or people may appear at various angles in the image, so the corresponding rotation weight may be high.

[0126] As an example, in order to further increase the efficiency of total contribution value calculation, the aforementioned S1051b may specifically include: for each recognition result, performing the following operations: A1 to A3 respectively.

[0127] A1: For each object in the recognition result, determine the target category that best matches the object from a preset category library. The category library includes multiple categories, each category corresponds to a rotation weight, and the target category is any one of the multiple categories.

[0128] As an example, the preset category library is a database containing multiple categories, each category represents a type of object, and is associated with a corresponding rotation weight to indicate the transmission rate of rotation action of the category object in the actual scene. In specific implementation, after obtaining the recognition result, for each object in the recognition result, the preset category library is traversed, and the features of the object are compared with the features of each category in the category library to find the category that best matches the object as the target category. Among them, the category library includes a collection of all recognizable object categories. As another example, classification algorithms, such as nearest neighbor, support vector machine or neural network algorithms, can also be used to find the category that best matches the object in the category library.

[0129] A2: For each target category, multiply the number of target objects corresponding to the target category by the rotation weight corresponding to the target category to obtain the contribution value of the target category to the recognition result. The number of target objects is the number of objects belonging to the target category in the recognition result.

[0130] As an example, for each target category, the number of target objects corresponding to the target category is multiplied by the rotation weight to obtain the contribution value of the target category to the recognition result. This contribution value reflects the importance of the object of the target category in determining the image direction.

[0131] A3: Add up multiple contribution values ​​to get the total contribution value of the recognition result.

[0132] Among them, the total contribution value is a comprehensive measure of the current recognition result, which indicates the credibility of the current recognition result.

[0133] In the embodiment of the present application, a unified recognition and evaluation standard is provided for all objects using a preset category library and rotation weights, which effectively reduces the complexity of setting weights for each object separately. In addition, through the preset category library, the recognized objects can be quickly classified and the corresponding rotation weights can be applied without calculating the weights for each object separately, which greatly improves the processing efficiency.

[0134] S1052b: Search for the second target recognition result with the highest total contribution value among the n recognition results.

[0135] It is worth mentioning here that when the total contribution values ​​of two or more recognition results are exactly the same, the directions of the highest identical scores are output in the order of priority of 0°, 180°, 270°, and 90°.

[0136] S1053b: Determine the direction of the target image corresponding to the second target recognition result as the actual direction of the image to be recognized.

[0137] In the embodiment of the present application, by introducing the rotation weight and the total contribution value, and combining the comprehensive evaluation of multiple recognition results, the actual direction of the image to be recognized can be determined more accurately. The rotation weight is determined based on the incidence of the object's rotation action in the actual scene, so that the actual direction of the image to be recognized that is finally determined can be closer to the actual situation, reducing the possibility of misjudgment of the direction due to a single recognition result error.

[0138] Next, the aforementioned S105 will be further explained with reference to a specific example.

[0139] First, objects can be divided into three categories based on the possibility of rotation or inversion: first-class objects, second-class objects, and third-class objects. Then, corresponding weight values ​​(W1, W2, W3) are assigned to the three different categories of objects to reflect the importance and differences of the three different categories of objects in specific situations. Specifically:

[0140] 1. Class I objects (extremely unlikely to rotate or invert, W1 = 100): refers to objects that are almost impossible to rotate or invert in daily life and natural environment, such as trains, cars, ships, fire hydrants, stop signs, etc.

[0141] 2. Class II objects (likely to rotate or invert, W2=10): refers to objects that have a certain possibility of rotating or inverting, such as people, animals, benches, text, charts, desktops, etc.

[0142] 3. Class III objects (flexible and changeable orientation, W3=1): refers to objects whose orientation is very easy to change or whose design allows multiple postures. For example, small objects such as spoons, toothbrushes, forks, scissors, etc. Ensure that when low-probability rotation and inverted objects are identified, they cannot be offset by an order of magnitude higher-probability rotation and inverted objects.

[0143] Among them, the highest weight value W1 (for example, W1=100) is set for the first category of objects, aiming to emphasize the importance of the first category of objects in image recognition and processing, and to ensure that even if misjudgment occurs, its impact will be effectively suppressed due to the extremely low probability and high weight setting. A medium weight value W2 (for example, W2=10) is set for the second category of objects, which not only reflects the possibility of rotation or inversion of the second category of objects, but also ensures that the importance of the second category of objects in the recognition process is not completely ignored. The lowest weight value W3 (for example, W3=1) is set for the third category of objects, aiming to reflect the uncertainty and flexibility of the direction of the third category of objects, and also ensures that during the recognition process, the recognition results of the third category of objects will not excessively affect the accuracy and reliability of the entire system.

[0144] By setting three different weights for the three types of objects, it can be ensured that during the recognition process, for objects with a low probability of being rotated or inverted (such as trains, cars, etc.), even if there is a misjudgment, it will not be offset by an order of magnitude higher probability of being rotated or inverted. This can improve the accuracy and reliability of object recognition when processing complex images containing multiple object types.

[0145] The position information includes the x, y, length and width information of the object in the image. That is, the rectangular frame seen in the object detection image indicates the location of the object. The position information is used to distinguish objects that are not upright in the training sample set, such as people lying on the grass.

[0146] By analyzing objects in non-perpendicular orientations in the training set, we can improve the model’s ability to recognize these objects without relying on image rotation augmentation training. For example, by observing a person lying on the grass, we can train the model to recognize objects even when they appear in non-typical orientations.

[0147] Further, the recognition process can be refined according to the aspect ratio of the object. Specifically, the objects are divided into two categories: one is an object whose length is greater than the width, such as a standing person; the other is an object whose width is greater than the length, such as a bench or a quadruped. Based on the above classification, the above two categories of objects can be further refined into a first subclass object and a second subclass object. Specifically, when the position information of the detected object does not match the aspect ratio feature of the category to which it belongs, the object is determined to be a first subclass object, thereby reducing the credibility of the recognition result in such a specific direction. On the contrary, when the position information of the detected object matches the aspect ratio feature of the category to which it belongs, the object is determined to be a second subclass object. Specifically, the weight value W21 of the first subclass object can be set to 0.5, indicating that there is a certain uncertainty in the direction judgment; the weight value W22 of the second subclass object can be set to 1, indicating that there is a high credibility in the direction judgment.

[0148] For example, if a person in a picture is recognized by the model when it is rotated 90° to the left or right, its exact orientation may not be distinguished without considering the position information. However, by combining the position information and the aspect ratio weight, we can get a lower weighted score when rotated 90° to the left or right, and a higher score in the positive direction, thereby helping the model to correctly identify the direction of the object. In this way, not only the accuracy of object detection is improved, but also the model's ability to recognize objects in different directions is enhanced, enabling it to distinguish the direction and posture of objects more finely.

[0149] As a specific example, the calculation formula of the total contribution value S is:

[0150] S=W1×CNT1+W2×W21×CNT2+W2×W22×CNT3+W3×CNT4.

[0151] In the formula, CNT1 represents the first number of objects of the first category, CNT2 represents the second number of objects of the first subcategory, CNT3 represents the third number of objects of the second subcategory, CNT4 represents the fourth number of objects of the third category, W1 represents the weight value of objects of the first category, W2 represents the weight value of objects of the second category, W21 represents the weight value of objects of the first subcategory, W22 represents the weight value of objects of the second subcategory, and W3 represents the weight value of objects of the third category.

[0152] Table 1 below lists several common image direction judgment situations:

[0153]

[0154] Table 1

[0155] In a specific implementation scenario, a picture to be identified is taken at 90° clockwise, and the picture to be identified is regarded as the reference 0°. The image features corresponding to 0°, 90°, 180° and 270° are respectively input into the above-mentioned object recognition model. The obtained recognition results are analyzed and it can be seen that when 0°, 90° and 180° are used as input angles for detection, the object is not successfully detected. However, when 270° is used as the input angle for detection, the object recognition model obtains the detection information of three objects. Specifically, these objects are highly matched with the target category in terms of aspect ratio and are classified as second subclass objects. According to the formula corresponding to the above-mentioned total contribution value S, the total contribution value corresponding to 270° is 3 (1×1×3=3), and the total contribution values ​​corresponding to the three angles are all 0. In summary, it can be concluded that the actual direction of the picture to be identified is the direction shown when the picture is rotated 270° clockwise.

[0156] Based on the image direction recognition method provided in the above embodiment, the present application also provides a specific implementation of an image direction recognition device. Please refer to the following embodiment.

[0157] Figure 8 The following is a schematic diagram showing the structure of a device for identifying the direction of an image provided by an embodiment of the present application. Figure 8 As shown, the image direction recognition device includes:

[0158] The first acquisition module 801 is used to acquire a picture to be identified;

[0159] The first rotation module 802 is used to rotate the image to be identified by n-1 preset angles based on a preset reference, to obtain n-1 reference images with different rotation directions, where n is a positive integer greater than or equal to 2;

[0160] The first extraction module 803 is used to perform image convolution operation on each target image to extract a corresponding set of image features, where the target image includes n-1 reference images and the image to be identified;

[0161] A first determination module 804 is used to input each set of image features into a pre-trained object recognition model to obtain a recognition result, where the recognition result includes the objects existing in the target image and the location information of the objects;

[0162] The second determination module 805 is used to determine the actual direction of the image to be identified among the n identification results according to each identification result.

[0163] In the embodiment of the present application, the first rotation module 802 and the first extraction module 803 are used to obtain n groups of image features of the image to be identified, so that the multi-angle image features of the image to be identified in different rotation directions can be captured. Then, the first determination module 804 is used to input these multi-angle image features into the pre-trained object recognition model, and the content learned by the object recognition model can be used to accurately analyze and identify the objects contained in the image to be identified in different directions. Furthermore, the second determination module 805 is used to analyze the recognition results of the image to be identified in multiple directions, and it can be evaluated which direction has the highest consistency and accuracy of the recognition results, thereby determining the actual direction of the image to be identified.

[0164] The device does not rely on the EXIF ​​information of the image or the gyroscope of the device, but judges the direction through the recognition result (object and position information of the object) obtained by the first determination module 804, that is, the image content. Specifically, the second determination module 805 can evaluate the direction of the object in the image from multiple angles through the recognition results of the image in different rotation directions, thereby bypassing the direction recognition deviation caused by hardware limitations or information loss, and effectively improving the accuracy of the direction recognition of the image to be recognized.

[0165] In some embodiments, in order to improve the computational efficiency and accuracy of image features, the first extraction module 803 may specifically include:

[0166] The first determination submodule is used to transform the target image using Fourier transform to obtain a frequency domain vector.

[0167] The first acquisition submodule is used to obtain the convolution kernel corresponding to the target image.

[0168] The second determination submodule is used to transform the convolution kernel to obtain a transformed convolution kernel.

[0169] The third determination submodule is used to perform a point multiplication between the frequency domain vector and the transformed convolution kernel to obtain an intermediate result.

[0170] The fourth determination submodule is used to perform an inverse Fourier transform on the intermediate result to obtain a convolution result.

[0171] The first extraction submodule is used to extract a set of image features representing the target image according to the convolution result.

[0172] In some embodiments, in order to accurately obtain the recognition result corresponding to each group of image features, the aforementioned device may further include:

[0173] The second acquisition module is used to acquire a training sample set, where the training sample set includes multiple training samples, each training sample includes a picture and a label corresponding to the picture, and the label includes objects included in the picture and location information corresponding to each object.

[0174] The training module is used to train the pre-built initial recognition model using multiple training samples until the training stop condition is met to obtain a trained object recognition model.

[0175] In some embodiments, in order to accurately identify and obtain the recognition results corresponding to each set of image features, the aforementioned training module may specifically include:

[0176] The second extraction submodule is used to perform, for each training sample, the following steps: inputting the image into the convolution layer of the convolutional neural network, and extracting the feature vector of the image using the rotation invariant convolution kernel introduced in the convolution layer.

[0177] The fifth determination submodule is used to perform nonlinear transformation on the feature vector using an activation function to obtain a transformed feature representation.

[0178] The sixth determination submodule is used to input the transformed feature representation into the fully connected layer, and use the fully connected layer to aggregate and classify the transformed features to obtain the recognition result of the image.

[0179] The seventh determination submodule is used to determine the loss function value of the initial recognition model according to the recognition results of each training sample.

[0180] The eighth determination submodule is used to obtain a trained object recognition model when the loss function value meets a preset training stop condition.

[0181] In some embodiments, in order to accurately identify and obtain the recognition results corresponding to each set of image features, the aforementioned second extraction submodule may specifically include:

[0182] The first calculation unit is used to calculate the gradient information of the image in the horizontal and vertical directions and construct a gradient map.

[0183] The second calculation unit is used to calculate the corresponding convolution kernel response for each position of the gradient map using the rotation invariant convolution kernel.

[0184] The construction unit is used to construct a feature map of the image using a gradient map and multiple convolution kernel responses, where the feature map includes feature information with rotation invariance.

[0185] The conversion unit is used to convert the feature map into a feature vector.

[0186] In some embodiments, in order to determine the actual direction of the image to be recognized, the second determination module may specifically include:

[0187] A first search submodule, used to search for a first target recognition result having the largest number of objects among the n recognition results;

[0188] The ninth determination submodule is used to determine the direction of the target image corresponding to the first target recognition result as the actual direction of the image to be recognized.

[0189] In some embodiments, in order to further improve the accuracy of determining the actual direction of the image to be identified, the second determination module may specifically include:

[0190] The tenth determination submodule is used to determine the total contribution value of each recognition result according to the rotation weight corresponding to each object in the recognition result, wherein the rotation weight is determined based on the occurrence rate of the rotation action of the object in the actual scene.

[0191] The second search submodule is used to search for a second target recognition result with the highest total contribution value among the n recognition results.

[0192] The eleventh determination submodule is used to determine the direction of the target image corresponding to the second target recognition result as the actual direction of the image to be recognized.

[0193] In some embodiments, in order to further increase the efficiency of total contribution value calculation, the second search submodule may specifically include:

[0194] The first determination unit is used to determine, for each object in the recognition result, a target category that best matches the object from a preset category library, the category library includes multiple categories, each category corresponds to a rotation weight, and the target category is any one of the multiple categories.

[0195] The second determination unit is used to multiply the number of target objects corresponding to each target category by the rotation weight corresponding to the target category to obtain a contribution value of the target category to the recognition result, where the number of target objects is the number of objects belonging to the target category in the recognition result.

[0196] The third determination unit is used to add the multiple contribution values ​​to obtain a total contribution value of the recognition result.

[0197] Fig. 9 The hardware structure diagram of the electronic device provided by the embodiment of the present application is shown. Based on the image direction recognition method provided by the above embodiment, the present application also provides a specific implementation method of the electronic device. Please refer to the following embodiments.

[0198] like Fig. 9 As shown, the electronic device may include a processor 901 and a memory 902 storing computer program instructions. When the processor executes the computer program instructions, any of the above-mentioned methods for identifying the direction of an image is implemented.

[0199] Specifically, the processor 901 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0200] The memory 902 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 902 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 902 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 902 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 902 is a non-volatile solid-state memory.

[0201] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0202] The processor 901 implements any one of the image direction recognition methods in the above embodiments by reading and executing computer program instructions stored in the memory 902 .

[0203] In one example, the electronic device may further include a communication interface 903 and a bus 910. Fig. 9 As shown, the processor 901, the memory 902, and the communication interface 903 are connected via a bus 910 and communicate with each other.

[0204] The communication interface 903 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0205] Bus 910 includes hardware, software or both, and the parts of electronic equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 910 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0206] The electronic device can execute the image direction recognition method in the embodiment of the present application, thereby realizing the combination Figure 1 and Figure 8 The described method and device for image direction recognition.

[0207] Based on the image direction recognition method provided in the above embodiment, the present application also provides a specific implementation of a computer-readable storage medium. The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, any of the above-mentioned image direction recognition methods is implemented.

[0208] Based on the image direction recognition method provided in the above embodiment, the present application also provides a specific implementation of a computer program product. The computer program product includes a computer program, and when the computer program is processed and executed, it implements any of the above-mentioned image direction recognition methods.

[0209] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0210] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0211] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0212] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0213] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for image direction recognition, characterized in that: The method comprises: Get the image to be identified; Based on the preset reference, the image to be identified is rotated by n-1 preset angles to obtain n-1 reference images with different rotation directions, where n is a positive integer greater than or equal to 2; Performing image convolution operations on each target image to extract a corresponding set of image features, wherein the target image includes n-1 reference images and the image to be identified; Inputting each group of the image features into a pre-trained object recognition model to obtain a recognition result, wherein the recognition result includes objects existing in the target image and location information of the objects; According to each of the recognition results, the actual direction of the image to be recognized is determined among the n recognition results.

2. The method according to claim 1, characterized in that The determining, according to each of the recognition results, the actual direction of the image to be recognized in the n recognition results includes: Searching for the first target recognition result having the largest number of objects among the n recognition results; The direction of the target image corresponding to the first target recognition result is determined as the actual direction of the image to be recognized.

3. The method according to claim 1, characterized in that The determining, according to each of the recognition results, the actual direction of the image to be recognized in the n recognition results includes: For each recognition result, determining a total contribution value of the recognition result according to a rotation weight corresponding to each object in the recognition result, wherein the rotation weight is determined based on an occurrence rate of a rotation action of the object in an actual scene; Searching for a second target recognition result with the highest total contribution value among the n recognition results; The direction of the target image corresponding to the second target recognition result is determined as the actual direction of the image to be recognized.

4. The method according to claim 3, characterized in that For each recognition result, determining the total contribution value of the recognition result according to the rotation weight corresponding to each object in the recognition result includes: For each recognition result, perform the following operations: For each object in the recognition result, determine a target category that best matches the object from a preset category library, wherein the category library includes a plurality of categories, each of the categories corresponds to a rotation weight, and the target category is any one of the plurality of categories; For each target category, multiply the number of target objects corresponding to the target category by the rotation weight corresponding to the target category to obtain a contribution value of the target category to the recognition result, where the number of target objects is the number of objects belonging to the target category in the recognition result; The multiple contribution values ​​are added together to obtain the total contribution value of the recognition result.

5. The method according to any one of claims 1 to 4, characterized in that: The image convolution operation is performed on each target image to extract a corresponding set of image features, including: For each target image, perform the following operations: Transforming the target image using Fourier transform to obtain a frequency domain vector; Obtaining a convolution kernel corresponding to the target image; Transforming the convolution kernel to obtain a transformed convolution kernel; Performing a point multiplication on the frequency domain vector and the transformed convolution kernel to obtain an intermediate result; Performing inverse Fourier transform on the intermediate result to obtain a convolution result; A set of image features representing the target image is extracted according to the convolution result.

6. The method according to any one of claims 1 to 4, characterized in that: Before inputting each group of the image features into a pre-trained object recognition model to obtain a recognition result, the method further includes: Acquire a training sample set, the training sample set comprising a plurality of training samples, each of the training samples comprising a picture and a label corresponding to the picture, the label comprising objects contained in the picture and location information corresponding to each of the objects; The pre-constructed initial recognition model is trained using the multiple training samples until a training stop condition is met, thereby obtaining a trained object recognition model.

7. The method according to claim 6, characterized in that The initial recognition model is constructed based on the architecture of a convolutional neural network, and the method further includes: Introducing a rotationally invariant convolution kernel into the convolutional layer of the convolutional neural network; The method of training the pre-built initial recognition model using the plurality of training samples until a training stop condition is met to obtain a trained object recognition model comprises: For each training sample, perform the following steps B1 to B3 respectively: B1: inputting the image into a convolutional layer of a convolutional neural network, and extracting a feature vector of the image using a rotationally invariant convolution kernel introduced in the convolutional layer; B2: Perform nonlinear transformation on the feature vector using an activation function to obtain a transformed feature representation; B3: Inputting the transformed feature representation into a fully connected layer, and using the fully connected layer to aggregate and classify the transformed features to obtain a recognition result of the image; Determining the loss function value of the initial recognition model according to the recognition results of each of the training samples; When the loss function value satisfies a preset training stop condition, a trained object recognition model is obtained.

8. The method according to claim 7, characterized in that The step of inputting the image into a convolutional layer of a convolutional neural network and extracting a feature vector of the image using a rotationally invariant convolution kernel introduced in the convolutional layer comprises: Calculate the gradient information of the image in the horizontal and vertical directions and construct a gradient map; For each position of the gradient map, a corresponding convolution kernel response is calculated using a rotationally invariant convolution kernel; Using the gradient map and the multiple convolution kernel responses, construct a feature map of the image, wherein the feature map includes feature information with rotation invariance; The feature map is converted into a feature vector.

9. A device for identifying the direction of an image, characterized in that: The device comprises: A first acquisition module is used to acquire a picture to be identified; A first rotation module is used to rotate the image to be identified by n-1 preset angles based on a preset reference, to obtain n-1 reference images with different rotation directions, where n is a positive integer greater than or equal to 2; A first extraction module is used to perform image convolution operation on each target image to extract a corresponding set of image features, wherein the target image includes n-1 reference images and the image to be identified; A first determination module is used to input each group of the image features into a pre-trained object recognition model to obtain a recognition result, wherein the recognition result includes objects existing in the target image and location information of the objects; The second determination module is used to determine the actual direction of the image to be identified among the n identification results according to each of the identification results.

10. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the image direction recognition method as described in any one of claims 1-8 is implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method for identifying the direction of an image according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that The invention comprises a computer program, wherein when the computer program is processed and executed, the method for recognizing the direction of an image according to any one of claims 1 to 8 is implemented.

Images