Image stretching recognition and model training method, device, equipment, medium and product

Abstract

The specification discloses an image stretching recognition and model training method, device, equipment, medium and product. The image stretching recognition method comprises the following steps: acquiring an image to be recognized; inputting the acquired image to be recognized into an image stretching recognition model to obtain a plurality of different types of feature data, and outputting an image stretching recognition result corresponding to the image to be recognized based on the plurality of different types of feature data; and the image stretching recognition model is trained based on a plurality of images with known state labels, wherein the state labels comprise stretching labels and non-stretching labels.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method, apparatus, device, medium and product for image stretching recognition and model training. Background Technology

[0002] With the popularization and rise of short video and live streaming technologies, related image and video content is experiencing explosive growth. Among this content, the cover image or video is a particularly important element to watch out for. However, due to the different cover processing methods used by various content providers, many image covers are deformed, especially stretched, which can cause visual discomfort to viewers. Therefore, there is an urgent need for a solution that can more accurately and efficiently identify image stretching. Summary of the Invention

[0003] This specification provides an image stretching recognition and model training method, apparatus, device, medium, and product. By combining multiple different types of feature data from the image to be recognized to jointly identify whether the image is stretched, the accuracy and efficiency of image stretching recognition are improved. The above technical solution is as follows:

[0004] Firstly, embodiments of this specification provide an image stretching detection method, including:

[0005] Acquire the image to be recognized;

[0006] The image to be identified is input into the image stretching recognition model to obtain multiple different types of feature data, and the image stretching recognition result corresponding to the image to be identified is output based on the multiple different types of feature data. The image stretching recognition model is trained on multiple images with known state labels, including stretching labels and non-stretching labels.

[0007] In one possible implementation, the image stretching recognition model described above includes a deep dual-branch convolutional neural network, which includes two different types of feature extraction branch networks.

[0008] The above-mentioned input of the image to be identified into the image stretching recognition model yields multiple different types of feature data, and outputs the image stretching recognition result corresponding to the image to be identified based on the multiple different types of feature data, including:

[0009] The image to be identified is input into the first feature extraction branch network of the deep dual-branch convolutional neural network, and the first feature data corresponding to the image to be identified is output.

[0010] The image to be identified is input into the second feature extraction branch network of the deep dual-branch convolutional neural network, and the second feature data corresponding to the image to be identified is output.

[0011] The first feature data and the second feature data are fused together to obtain fused feature data;

[0012] Based on the above fused feature data, the image stretching recognition result corresponding to the above image to be recognized is output.

[0013] In one possible implementation, both the first feature data and the second feature data mentioned above include a long dimension, a wide dimension, and a channel dimension.

[0014] The above-mentioned first feature data and the above-mentioned second feature data are fused to obtain fused feature data, including:

[0015] The first feature data and the second feature data are multiplied by a matrix to eliminate the long and wide dimensions of the first feature data and the second feature data, resulting in the fused feature data; the fused feature data includes the channel dimension.

[0016] In one possible implementation, the first feature extraction branch network of the aforementioned deep dual-branch convolutional neural network is an EfficientNet network, and the second feature extraction branch network of the aforementioned deep dual-branch convolutional neural network is a Spatial CNN network.

[0017] In one possible implementation, before inputting the image to be recognized into the image stretching recognition model, the method further includes:

[0018] The image to be identified is rotated to obtain multiple rotated images corresponding to the image to be identified. Each of the multiple rotated images has a different rotation angle.

[0019] The above-mentioned input of the image to be identified into the image stretching recognition model yields multiple different types of feature data, and outputs the image stretching recognition result corresponding to the image to be identified based on the multiple different types of feature data, including:

[0020] The rotated image corresponding to the above image to be identified is input into the image stretching recognition model to obtain multiple different types of feature data, and the image stretching recognition result corresponding to the above rotated image is output based on the multiple different types of feature data.

[0021] Based on the image stretching recognition results corresponding to each of the above-mentioned multiple rotated images, the image stretching recognition result of the above-mentioned image to be recognized is determined.

[0022] In one possible implementation, the image stretching recognition result includes the stretching probability;

[0023] After inputting the image to be identified into the image stretching recognition model to obtain multiple different types of feature data, and outputting the image stretching recognition result corresponding to the image to be identified based on the multiple different types of feature data, the above method further includes:

[0024] Determine whether the above stretching probability is greater than the stretching determination threshold;

[0025] If so, then the image to be identified is determined to be a stretched image;

[0026] If not, then the image to be identified is determined to be a non-stretched image.

[0027] In one possible implementation, after acquiring the image to be recognized and before inputting the image to be recognized into the image stretching recognition model, the method further includes:

[0028] The image to be recognized is scaled proportionally and filled into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts to obtain the image to be recognized that meets the preset size.

[0029] Secondly, embodiments of this specification provide a method for training an image stretching recognition model, including:

[0030] Acquire multiple images with known state labels, including stretched and unstretched labels;

[0031] The image with the known state label is input into the image stretching recognition model to obtain multiple different types of feature data, and the predicted image stretching recognition result corresponding to the image with the known state label is output based on the multiple different types of feature data.

[0032] The image stretching recognition model is updated based on the predicted image stretching recognition results and status labels corresponding to the multiple images with known status labels. The image stretching recognition model is used to implement the image stretching recognition method in the first aspect or any possible implementation of the first aspect of the embodiments of this specification.

[0033] In one possible implementation, acquiring multiple images with known state labels includes:

[0034] Acquire multiple baseline images;

[0035] Determine the image processing type corresponding to each of the above reference images; the above image processing types include non-stretching processing type, height stretching processing type, and width stretching processing type.

[0036] The above reference image is stretched according to the corresponding image processing type and stretching ratio to obtain the processed image;

[0037] The image with a stretching ratio greater than the stretching threshold is set as the image corresponding to the stretching label.

[0038] The processed image with a stretching ratio not greater than the stretching threshold is set as the image corresponding to the non-stretched label.

[0039] In one possible implementation, acquiring multiple images with known state labels includes:

[0040] Obtain multiple raw images with known state labels;

[0041] The original image is processed to obtain a first processed image; the first processed image is an image of part or all of the original image.

[0042] The first processed image is scaled proportionally and filled into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts to obtain an image with known status labels.

[0043] In one possible implementation, the first processed image is scaled proportionally and filled into a preset-size template. If there are unfilled portions in the preset-size template, preset pixels are filled into these unfilled portions to obtain an image with known state labels, including:

[0044] The first processed image is scaled proportionally and filled into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts, and a portion of the first processed image is covered with the preset pixels to obtain an image with a known state label.

[0045] Thirdly, embodiments of this specification provide an image stretching recognition device, comprising:

[0046] The first acquisition module is used to acquire the image to be recognized;

[0047] The recognition module is used to input the image to be recognized into the image stretching recognition model to obtain multiple different types of feature data, and output the image stretching recognition result corresponding to the image to be recognized based on the multiple different types of feature data; the image stretching recognition model is trained based on multiple images with known state labels, and the state labels include stretching labels and non-stretching labels.

[0048] In one possible implementation, the image stretching recognition model described above includes a deep dual-branch convolutional neural network, which includes two different types of feature extraction branch networks.

[0049] The aforementioned identification module includes:

[0050] The first feature extraction unit is used to input the image to be identified into the first feature extraction branch network of the deep dual-branch convolutional neural network and output the first feature data corresponding to the image to be identified.

[0051] The second feature extraction unit is used to input the image to be identified into the second feature extraction branch network of the deep dual-branch convolutional neural network and output the second feature data corresponding to the image to be identified.

[0052] The fusion unit is used to fuse the first feature data and the second feature data to obtain fused feature data;

[0053] The output unit is used to output the image stretching recognition result corresponding to the image to be recognized based on the above-mentioned fused feature data.

[0054] In one possible implementation, both the first feature data and the second feature data mentioned above include a long dimension, a wide dimension, and a channel dimension.

[0055] The aforementioned fusion unit is specifically used for:

[0056] The first feature data and the second feature data are multiplied by a matrix to eliminate the long and wide dimensions of the first feature data and the second feature data, resulting in the fused feature data; the fused feature data includes the channel dimension.

[0057] In one possible implementation, the first feature extraction branch network of the aforementioned deep dual-branch convolutional neural network is an EfficientNet network, and the second feature extraction branch network of the aforementioned deep dual-branch convolutional neural network is a Spatial CNN network.

[0058] In one possible implementation, the image stretching recognition device further includes:

[0059] The generation module is used to rotate the above image to be identified to obtain multiple rotated images corresponding to the above image to be identified, and the rotation angles of the multiple rotated images are different.

[0060] The aforementioned identification module is specifically used for:

[0061] The rotated image corresponding to the above image to be identified is input into the image stretching recognition model to obtain multiple different types of feature data, and the image stretching recognition result corresponding to the above rotated image is output based on the multiple different types of feature data.

[0062] Based on the image stretching recognition results corresponding to each of the above-mentioned multiple rotated images, the image stretching recognition result of the above-mentioned image to be recognized is determined.

[0063] In one possible implementation, the image stretching recognition result includes the stretching probability;

[0064] The aforementioned image stretching recognition device also includes a judgment module, used for:

[0065] Determine whether the above stretching probability is greater than the stretching determination threshold;

[0066] If so, then the image to be identified is determined to be a stretched image;

[0067] If not, then the image to be identified is determined to be a non-stretched image.

[0068] In one possible implementation, the image stretching recognition device further includes:

[0069] The preprocessing module is used to scale the image to be recognized proportionally and fill it into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts to obtain the image to be recognized that meets the preset size.

[0070] Fourthly, embodiments of this specification provide an image stretching recognition model training device, comprising:

[0071] The second acquisition module is used to acquire multiple images with known state labels; the state labels include stretched labels and non-stretched labels.

[0072] The first training module is used to input the image with the known state label into the image stretching recognition model to obtain multiple different types of feature data, and output the predicted image stretching recognition result corresponding to the image with the known state label based on the multiple different types of feature data.

[0073] The second training module is used to update the image stretching recognition model based on the predicted image stretching recognition results and state labels corresponding to the multiple images with known state labels; the image stretching recognition model is used to implement the image stretching recognition method in the first aspect or any possible implementation of the first aspect of the embodiments of this specification.

[0074] In one possible implementation, the second acquisition module described above includes:

[0075] The first acquisition unit is used to acquire multiple reference images;

[0076] The type determination unit is used to determine the image processing type corresponding to each of the above multiple reference images; the above image processing types include non-stretching processing type, height stretching processing type and width stretching processing type.

[0077] The stretching processing unit is used to stretch the above reference image according to the corresponding image processing type and stretching ratio to obtain the processed image.

[0078] The first label unit is used to set the processed image with a stretching ratio greater than the stretching threshold as the image corresponding to the stretching label.

[0079] The second label unit is used to set the processed image with a stretching ratio not greater than the stretching threshold as the image corresponding to the non-stretched label.

[0080] In one possible implementation, the second acquisition module described above includes:

[0081] The second acquisition unit is used to acquire multiple original images with known state labels;

[0082] The first processing unit is used to process the original image to obtain a first processed image; the first processed image is an image of part or all of the original image.

[0083] The second processing unit is used to scale the first processed image proportionally and fill it into a preset size template. If there are unfilled parts in the preset size template, the unfilled parts are filled with preset pixels to obtain an image with a known state label.

[0084] In one possible implementation, the second processing unit described above is specifically used for:

[0085] The first processed image is scaled proportionally and filled into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts, and a portion of the first processed image is covered with the preset pixels to obtain an image with a known state label.

[0086] Fifthly, embodiments of this specification provide an electronic device, including: a processor and a memory;

[0087] The processor is connected to the memory.

[0088] The aforementioned memory is used to store executable program code;

[0089] The processor reads the executable program code stored in the memory to run the program corresponding to the executable program code, so as to execute the method provided by the first aspect or any possible implementation of the first aspect or the second aspect or any possible implementation of the embodiments of this specification.

[0090] Sixthly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the method provided by the first aspect or any possible implementation of the first aspect or the second aspect or any possible implementation of the second aspect of the embodiments of this specification.

[0091] In a seventh aspect, embodiments of this specification provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to perform the method provided by the first aspect or any possible implementation of the first aspect or the second aspect or any possible implementation of the second aspect of the embodiments of this specification.

[0092] This specification describes an embodiment that acquires an image to be identified; inputs the image to be identified into an image stretching recognition model to obtain multiple different types of feature data; and outputs the image stretching recognition result corresponding to the image to be identified based on these multiple different types of feature data. The image stretching recognition model is trained on multiple images with known state labels, including stretching labels and non-stretching labels. By extracting and combining multiple different types of feature data from the image to be identified, the correlation between features can be better captured and utilized, improving the accuracy and efficiency of image stretching recognition. Attached Figure Description

[0093] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0094] Figure 1 A schematic diagram of the architecture of an image stretching recognition system provided for an exemplary embodiment of this specification;

[0095] Figure 2 A flowchart illustrating an image stretching recognition method provided as an exemplary embodiment of this specification;

[0096] Figure 3 This is a schematic diagram illustrating an implementation process for generating an image of a preset size to be recognized, provided as an exemplary embodiment of this specification.

[0097] Figure 4 A schematic diagram of the structure of a deep dual-branch convolutional neural network is provided for an exemplary embodiment of this specification;

[0098] Figure 5 A schematic diagram illustrating the test results of an image stretching recognition test provided as an exemplary embodiment of this specification;

[0099] Figure 6 A schematic diagram of an image stretching recognition result provided for an exemplary embodiment of this specification;

[0100] Figure 7 A schematic flowchart of another image stretching recognition method provided for an exemplary embodiment of this specification;

[0101] Figure 8 A schematic diagram illustrating the implementation process of an image stretching recognition method provided as an exemplary embodiment of this specification;

[0102] Figure 9 A flowchart illustrating an image stretching recognition model training method provided as an exemplary embodiment of this specification;

[0103] Figure 10 This is a schematic diagram illustrating an implementation process for generating an image with known state labels, provided as an exemplary embodiment of this specification.

[0104] Figure 11 A schematic diagram illustrating another implementation process for generating an image with known state labels, provided as an exemplary embodiment of this specification;

[0105] Figure 12 A schematic diagram of the structure of an image stretching recognition device provided for an exemplary embodiment of this specification;

[0106] Figure 13 A schematic diagram of the structure of an image stretching recognition model training device provided as an exemplary embodiment of this specification;

[0107] Figure 14 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this specification. Detailed Implementation

[0108] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0109] The terms "first," "second," "third," etc., used in the description, claims, and accompanying drawings are used to distinguish different objects and not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0110] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the images to be identified and reference images involved in this specification were obtained under full authorization.

[0111] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0112] Aspect Ratio: In this specification, it refers to the ratio of the image's width to its height. For example, if an image is 1200 pixels wide and 800 pixels high, then the aspect ratio of the image is 1.5 (1200 divided by 800).

[0113] Proportional scaling: In this specification, it refers to maintaining the aspect ratio of an image while resizing it. That is, the width and height of the image are scaled proportionally. Proportional scaling preserves the original proportions and shape of the image, preventing distortion.

[0114] Reference image: In this specification, it refers to an image taken directly by an image acquisition device such as a mobile phone or camera in a natural environment, without being stretched or otherwise processed, as well as an image scaled up proportionally.

[0115] Image stretching: In this specification, stretching a reference image in one or both of the horizontal or vertical directions results in an image with a different aspect ratio from the corresponding reference image. Specifically, stretching the reference image simultaneously in two directions to obtain an image with the same aspect ratio as the reference image is called proportional scaling.

[0116] Stretch ratio: In this specification, it refers to the ratio of the stretched dimension of the reference image to the size of the reference image. When calculating the stretch ratio, the image should first be scaled proportionally so that one side of it matches the corresponding reference image. For example, if the image size is 200×240 and the corresponding reference image size is 100×80, the image after proportional scaling will be 100×120. Compared to the reference image, this is a height stretch (vertical stretch), and the stretch ratio is (120-80) / 80 = 0.5.

[0117] The present specification will now be described in detail with reference to specific embodiments.

[0118] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the architecture of an image stretching recognition system provided as an exemplary embodiment of this specification. Figure 1 As shown, the image stretching recognition system may include a terminal 110 and a server 120.

[0119] Terminal 110 is connected to server 120 via network 130. Optionally, terminal 110 may be a smartphone, tablet, laptop, desktop computer, smartwatch, etc., but is not limited to these. Terminal 110 has an application installed and running that supports image stretching recognition. Network 130 may be a wireless network or a wired network.

[0120] Server 120 is an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. Server 120 provides background services for applications running on terminal 110. For example, server 120 provides corresponding services for applications running on terminal 110. In the embodiments of this specification, server 120 provides background services for applications running on the terminal that support image stretching recognition, such as training an image stretching recognition model.

[0121] Those skilled in the art will understand that the number of terminals 110 and servers 120 can be more or less. For example, there may be only one terminal 110 and one server 120, or there may be dozens or hundreds of terminals 110 and servers 120, or even more. In this case, the system architecture may also include other terminals and servers. The embodiments in this specification do not limit the number of terminals or the type of devices.

[0122] After introducing the system architecture of the embodiments of this specification, the application scenarios of the embodiments of this specification will be described below in conjunction with the above system architecture. In the following description, the terminal is also the terminal 110 in the above system architecture, and the server is also the server 120 in the above system architecture. The technical solutions provided by the embodiments of this specification can be applied to various image stretching recognition scenarios. For example, the technical solutions provided by the embodiments of this specification can be applied to the image stretching recognition of short video covers, or to the image stretching recognition of live program covers, or to the image stretching recognition of article illustrations.

[0123] Taking the application of the technical solution provided in the embodiments of this specification to the image stretching recognition of short video covers as an example, the terminal uploads an image of the short video cover to the server, and the server obtains the image of the short video cover. The server inputs the obtained image into the image stretching recognition model to obtain multiple different types of feature data. Based on the above multiple different types of feature data, the server outputs the image stretching recognition result corresponding to the image. The image stretching recognition model is trained on multiple images with known state labels, including stretching labels and non-stretching labels.

[0124] It should be noted that the above description is based on the application of the technical solution provided in the embodiments of this specification to the image stretching recognition of short video covers. For other scenarios, the implementation process belongs to the same inventive concept as described above, and the implementation process will not be repeated. The execution entity corresponding to the image stretching recognition method in one or more embodiments of this specification can be the aforementioned terminal 110, and the server 120 can be responsible for training the image stretching recognition model and sending it to the terminal 110; the execution entity corresponding to the image stretching recognition method in one or more embodiments of this specification can also be the server 120, and the terminal can be responsible for uploading the image to be recognized to the server. The specific implementation depends on the actual application environment, and this specification does not limit this aspect.

[0125] Next, combine Figure 1 This document describes the image stretching recognition method provided in the embodiments of this specification. Please refer to [link / reference needed] for details. Figure 2 This is a schematic flowchart illustrating an image stretching recognition method provided in an exemplary embodiment of this specification. Figure 2 As shown, with the server as the execution subject, this image stretching recognition method includes the following steps:

[0126] S202, Obtain the image to be recognized.

[0127] Specifically, the image to be recognized is the image that needs to be stretched for recognition. Therefore, before performing image stretching recognition, the image to be recognized needs to be acquired first. This image can be an image stored in the terminal's storage device, or an image captured by an image acquisition device such as the terminal's camera. For example, when a user creates a short video cover, they need to upload the cover image to the server. When setting the short video cover, the server will perform image stretching recognition to determine whether the uploaded cover image is stretched or distorted. The user can choose to select an image from the gallery or directly capture an image using the camera and upload it.

[0128] It is understood that the image to be identified can be a single image or multiple images. The server can obtain the images to be identified and process them one by one, or process them in parallel. For ease of explanation, this specification uses the image stretching and recognition process of a single image to be identified as an example.

[0129] Optionally, after acquiring the image to be recognized but before stretching it into the recognition model, the image to be recognized can be scaled proportionally and filled into a preset-size template. If there are unfilled portions in the filled preset-size template, preset pixels can be filled into these unfilled portions to obtain an image to be recognized that meets the preset size. By scaling the image to be recognized proportionally and filling it into the preset size, it is possible to ensure that the input images received by the model have a consistent size, thus facilitating model processing; on the other hand, it also allows the model to better adapt to and process images of various sizes, thereby enhancing the model's robustness.

[0130] For example, such as Figure 3 As shown, the original image 310 to be recognized has a size of 330×725. It can be proportionally scaled to fill a preset size template (384×384) to output a preset size image 320, which has a size of 384×384. The proportional scaling ratio can be obtained using the following steps:

[0131]

[0132]

[0133] scale = max(scale_h, scale_w)

[0134] Where h and w represent the actual height and width of the original image 310, respectively; fix_h and fix_w represent the height and width of the preset size image 320, respectively; scale_h and scale_w represent the scaling ratios corresponding to the height and width, respectively; and scale is the final selected scaling ratio.

[0135] Understandably, the scaling factor in the proportional scaling step is the largest of the scaling factors corresponding to the height and width, so that at least one side of the original image 310 meets the preset size after scaling. If the aspect ratio of the original image 310 differs from the aspect ratio of the preset size, the original image 310, even after proportional scaling, may not completely fill the preset size template. Figure 3 The unfilled portion 330 is shown. At this time, the unfilled portion 330 can be filled with preset pixels, such as RGB (128,128,128) gray pixels, which can ensure that all images completely occupy the preset size, further ensuring the input consistency of the model.

[0136] It should be noted that the preset size and preset pixels mentioned above are merely illustrative examples, and the specific values ​​can be set according to the specific implementation situation, and are not limited here. In addition, the preset size template is a descriptive form for more conveniently illustrating the above proportional scaling and filling steps, and is not limited to a physical template.

[0137] S204: Input the image to be recognized into the image stretching recognition model to obtain multiple different types of feature data, and output the image stretching recognition result corresponding to the image to be recognized based on the multiple different types of feature data.

[0138] Specifically, this image stretching recognition model is trained on multiple images with known state labels, including stretching labels and non-stretching labels. Optionally, a stretching label indicates that the image is significantly stretched (the stretching ratio exceeds a stretching threshold), which causes visual discomfort; a non-stretching label indicates that the image is not stretched or slightly stretched (the stretching ratio does not exceed the stretching threshold), which does not cause visual discomfort. The stretching threshold can be set according to the specific implementation scenario and is not limited here. For example, if the stretching threshold is set to 0.6, then images with a stretching ratio greater than 0.6 are designated as images corresponding to the stretching label, and images with a stretching ratio less than 0.6 are designated as images corresponding to the non-stretching label.

[0139] This image stretching recognition model comprises a multi-branch deep learning network. Different branches can have different feature focuses and can extract different types of feature data. After the image to be recognized is input into the image stretching recognition model, it can extract multiple different types of feature data. Then, based on these feature data, subsequent feature recognition steps are performed to obtain the image stretching recognition result. Since the model is trained, it can learn from the training data how to determine whether an image is stretched, thus accurately identifying whether the image to be recognized is in a stretched state.

[0140] Optional, such as Figure 4As shown, to further optimize the efficiency and accuracy of the image stretching recognition model, the model includes a deep dual-branch convolutional neural network (DBN), which comprises two different types of feature extraction branch networks. The image to be recognized is input into the first feature extraction branch network of the DBN, which outputs first feature data corresponding to the image to be recognized. The image to be recognized is then input into the second feature extraction branch network of the DBN, which outputs second feature data corresponding to the image to be recognized. The first and second feature data are fused to obtain fused feature data. Based on the fused feature data, the image stretching recognition result corresponding to the image to be recognized is output.

[0141] Specifically, a deep bilinear convolutional neural network (DBCNN) can be used to extract features from the image to be recognized. DBCNN has two different types of feature extraction branches, which can extract different types of feature data respectively. For example, one branch extracts global features of the image to be recognized, and the other extracts local features. This allows the image stretching recognition model to extract richer feature data. Through feature data fusion, higher accuracy is achieved when recognizing stretched images. Furthermore, using a deep bilinear convolutional neural network results in lower model complexity, lower inference time, and higher efficiency while maintaining good recognition accuracy.

[0142] Optionally, the first feature extraction branch network of the aforementioned deep dual-branch convolutional neural network can be an EfficientNet network, and the second feature extraction branch network of the aforementioned deep dual-branch convolutional neural network can be a Spatial CNN network.

[0143] EfficientNet and Spatial CNN (SCNN) are two different types of convolutional neural networks. EfficientNet is a convolutional neural network that balances the depth, width, and resolution of the network. It finds a new scaling method through Neural Architecture Search (NAS), which can more comprehensively expand the various dimensions of the convolutional neural network. This allows EfficientNet to achieve very high accuracy while maintaining a low model size and computational cost. SCNN is a convolutional neural network specifically designed to handle spatial correlations in images. SCNN introduces a novel layer type, the spatial convolutional layer, which can directly process horizontal and vertical relationships in images, making it suitable for processing images containing rich spatial information.

[0144] In the embodiments of this specification, the EfficientNet network can be used to extract high-level and global features, while the SCNN network can be used to extract spatial information in the image. The combination of these two networks enables the image stretching recognition model to understand the image from multiple perspectives, thereby improving the accuracy of image stretching recognition.

[0145] For example, in this embodiment, the Deep Dual-Branch Convolutional Neural Network (DBCNN) uses two branches, EfficientNet and SCNN, for feature extraction. Both branches perform downsampling operations using convolutions with a stride of 2 to reduce the spatial dimensionality of the image to be recognized. This downsampling process significantly reduces computational and memory requirements while preserving important image features, accelerating model training and inference. In this process, each branch undergoes four rounds of convolutional downsampling to continuously extract and refine feature maps of the image to be recognized. These feature maps capture various information about the image, such as color, texture, shape, and structure. After four rounds of convolutional downsampling, the feature maps generated by the EfficientNet and SCNN branches are fused, allowing the model to benefit from the different types of features captured by both branches, thereby improving model performance.

[0146] In this example, DBCNN (Group A), which uses both EfficientNet and SCNN branches, was compared with DBCNN (Group B), which uses both VGG (Visual Geometry Group) and SCNN branches, in an image stretching recognition test. Under the same test conditions, the test results are as follows: Figure 5 As shown in the figure. Test results show that DBCNN, which uses both EfficientNet and SCNN branches, has higher precision and recall. Optionally, the first feature extraction branch network, EfficientNet, can be the EfficientNet-B3 version, which has better feature extraction capabilities than other versions of EfficientNet (such as EfficientNet-B1 and EfficientNet-B2).

[0147] Optionally, the first and second feature data extracted by the aforementioned deep dual-branch convolutional neural network both include length dimension, width dimension, and channel dimension; performing matrix multiplication on the first and second feature data can eliminate the length and width dimensions of the first and second feature data, resulting in fused feature data that retains more of the channel dimension.

[0148] Specifically, performing matrix multiplication directly on the two feature maps extracted by a deep bibranch convolutional neural network makes parallel computation easier, thus accelerating model training. Furthermore, eliminating the length and width dimensions reduces the number of parameters, thereby decreasing model complexity and computational cost. Additionally, matrix multiplication can calculate correlations through the inner product of feature data, thus better preserving the relationships between feature data and improving model accuracy.

[0149] For example, the structure of a deep bibranch convolutional neural network can be controlled so that the length and width dimensions of the first and second feature data output by the two branches are the same (as in the example above, the feature maps output by both branches of the deep bibranch convolutional neural network are downsampled by 16 times). Then, the first and second feature data can be reshaped into matrices of (length * width, channel) form, such as matrix A (channel 1, length * width) for the first feature data and matrix B (channel 2, length * width) for the second feature data. Next, matrices A and B can be multiplied by matrix multiplication (e.g., AB). T This yields a matrix C (channel 1, channel 2) corresponding to the fused feature data. In this way, the original long and wide dimensions can be eliminated through matrix multiplication, retaining only the channel dimension, reducing the computational complexity of the model while fully utilizing the channel information of the feature map. Finally, the matrix C (channel 1, channel 2) corresponding to the fused feature data can be input into the fully connected layer of a deep dual-branch convolutional neural network for classification prediction to obtain the image stretching recognition result. Additionally, in some embodiments, the matrix C corresponding to the fused feature data needs to be flattened into a one-dimensional form to meet the input requirements of the fully connected layer.

[0150] It should be noted that the above-described steps for fusing the first and second feature data are merely an exemplary and optional solution. In some specific embodiments, other matrix multiplication calculation steps or other feature fusion methods (such as feature concatenation) may also be used, and information in the long and / or wide dimensions may not be eliminated, as long as the feature fusion steps of the deep dual-branch convolutional neural network described in this specification can be achieved.

[0151] Optionally, the image stretching recognition model outputs the image stretching recognition result corresponding to the image to be recognized, including the stretching probability; after the image stretching recognition model outputs the image stretching recognition result, it is determined whether the stretching probability corresponding to the image stretching recognition result is greater than the stretching determination threshold; if so, the image to be recognized is determined to be a stretched image; if not, the image to be recognized is determined to be a non-stretched image.

[0152] Specifically, after inputting the image to be recognized into the image stretching recognition model, multiple different types of feature data are obtained. Feature fusion is then performed based on these different types of feature data. The fused feature data is input into a fully connected layer for classification prediction, and the output is then fed into an activation function (such as Sigmoid, Softmax, etc.) to map the output to (0,1), obtaining the stretching probability. This stretching probability is the image stretching recognition result. This stretching probability represents the probability of a positive class. Depending on the training of the image stretching recognition model, it can represent the probability that the image to be recognized is in a stretched state or the probability that it is not stretched. Finally, by setting a stretching determination threshold (such as 0.5 or 0.6), the above stretching probabilities are classified to obtain the final recognition result. This stretching determination threshold can be specifically set according to the specific implementation scenario and is not limited here.

[0153] For example, an image to be recognized is input into an image stretching recognition model, and the model performs classification prediction. The output result is then input into the Sigmoid function:

[0154]

[0155]

[0156] Where p represents the stretching probability, and pre represents the final recognition result of the image stretching recognition model, including two final recognition results: a stretched image and a non-stretched image. In this embodiment, the stretching determination threshold is set to 0.5. When the stretching probability p is greater than 0.5, the image to be recognized is identified as a stretched image; when the stretching probability is not greater than 0.5, the image to be recognized is identified as a non-stretched image.

[0157] For example, such as Figure 6 As shown, Figure 6 The image to be identified (a) is input into the image stretching recognition model, and the output stretching probability is 0.9982 (>0.5), indicating that the image to be identified (a) is a stretched image; Figure 6 The image to be identified (b) is input into the image stretching recognition model, and the output stretching probability is 0.0102 (<0.5), indicating that the image to be identified (b) is an unstretched image.

[0158] This specification describes an embodiment that acquires an image to be identified; inputs the image to be identified into an image stretching recognition model to obtain multiple different types of feature data; and outputs the image stretching recognition result corresponding to the image to be identified based on these multiple different types of feature data. The image stretching recognition model is trained on multiple images with known state labels, including stretching labels and non-stretching labels. By extracting and combining multiple different types of feature data from the image to be identified, the correlation between features can be better captured and utilized, improving the accuracy and efficiency of image stretching recognition.

[0159] Please refer to the following. Figure 7 This is a schematic flowchart illustrating another image stretching recognition method provided in an exemplary embodiment of this specification. Figure 7 As shown, the image stretching recognition method includes the following steps:

[0160] S702, acquire the image to be recognized.

[0161] Specifically, the steps in S702 are the same as those in S202, and will not be repeated here.

[0162] S704, rotate the image to be recognized to obtain multiple rotated images corresponding to the image to be recognized.

[0163] Specifically, such as Figure 8 As shown, the image to be recognized can be rotated at different angles, such as rotating once every 90°. This yields multiple rotated images of the image to be recognized, each corresponding to a specific rotation angle, and these rotated images have different rotation angles. The specific rotation angle and the number of rotated images can be set according to the specific implementation scenario, and are not limited here.

[0164] S706: Input the rotated image corresponding to the image to be recognized into the image stretching recognition model to obtain multiple different types of feature data, and output the image stretching recognition result corresponding to the rotated image based on the multiple different types of feature data.

[0165] Specifically, such as Figure 8 As shown, the rotated images corresponding to the images to be recognized are input into the image stretching recognition model for image stretching recognition, and an image stretching recognition result (such as stretching probability) corresponding to the rotated image can be obtained respectively. The steps of the image stretching recognition model to calculate the image stretching recognition result corresponding to the rotated image are the same as steps S204, and will not be repeated here.

[0166] S708, based on the image stretching recognition results corresponding to each of the multiple rotated images, determines the image stretching recognition result of the image to be recognized.

[0167] Specifically, such as Figure 8 As shown, by rotating the same image to be recognized multiple times and inputting it into the image stretching recognition model, multiple possible recognition results can be obtained. This reduces the error of a single recognition and improves the accuracy of the image stretching recognition model. This is because each rotation of the image to be recognized may expose different image features for the model to recognize, and the model may be more accurate in recognizing certain features. Finally, the results of the multiple recognitions can be fused using statistical methods, such as voting or probability averaging, to obtain a more robust recognition result as the image stretching recognition result of the image to be recognized.

[0168] For example, a voting mechanism can be used to determine the image stretching recognition result of the image to be recognized. The stretching probabilities of each of the multiple rotated images corresponding to the image to be recognized are counted. The number of stretching probabilities greater than the stretching judgment threshold and the number less than the stretching judgment threshold are compared, and the class with the highest number is taken as the image stretching recognition result of the image to be recognized. If there are cases where the number is the same, one class can be selected as the image stretching recognition result of the image to be recognized, depending on the specific implementation scenario. For example, if the image to be recognized corresponds to four rotated images, and the stretching probabilities of two rotated images are greater than the stretching judgment threshold, while the stretching probabilities of two rotated images are less than the stretching judgment threshold, then the final image stretching recognition result is considered to have a stretching probability greater than the stretching judgment threshold.

[0169] For example, the image stretching recognition result of the image to be recognized can be determined by probability averaging. The stretching probabilities of multiple rotated images corresponding to the image to be recognized are statistically analyzed, and the arithmetic mean of these stretching probabilities is taken as the image stretching recognition result of the image to be recognized. For instance, if the image to be recognized corresponds to four rotated images, and their respective stretching probabilities are 0.5298, 0.4987, 0.4896, and 0.4936, their arithmetic mean is 0.5029, which is the image stretching recognition result of the image to be recognized.

[0170] This specification describes an embodiment that acquires an image to be recognized; rotates the image to be recognized to obtain multiple rotated images; inputs these rotated images into an image stretching recognition model to obtain multiple different types of feature data, and outputs the image stretching recognition result for each rotated image based on these feature data; finally, based on the image stretching recognition results of these rotated images, the final stretching recognition result of the image to be recognized is determined. This can reduce recognition errors caused by image angles or certain specific features, thereby improving recognition accuracy; in addition, by comparing and deciding on multiple recognition results, the error that may occur with a single recognition result can be reduced, improving the reliability of the final recognition result.

[0171] Next, combine Figure 1Taking the image stretching recognition model training performed by server 120 as an example, this specification introduces an exemplary embodiment of an image stretching recognition model training method. Figure 9 As shown, the training method for this image stretching recognition model includes the following steps:

[0172] S902, acquire multiple images with known state labels.

[0173] Specifically, before performing image stretching recognition, an image stretching recognition model can be trained using server 120. The trained model is then deployed on terminal 110 or server 120 to perform image stretching recognition on the image to be recognized. When training the image stretching recognition model, server 120 can acquire multiple images with status labels via the network. These status labels include stretching labels and non-stretching labels. These status labels characterize the image stretching state corresponding to the image. Optionally, a stretching label indicates that the image is in a significantly stretched state, which will cause visual discomfort; a non-stretching label indicates that the image is in a non-stretched or slightly stretched state, which will not cause visual discomfort.

[0174] Optionally, the image with the known status label can be obtained by processing a reference image. The specific steps include: acquiring multiple reference images; determining the image processing type corresponding to each of the multiple reference images; the image processing type includes non-stretching processing type, height stretching processing type, and width stretching processing type; stretching the reference images according to the corresponding image processing type and stretching ratio to obtain the processed image; setting the processed image with the stretching ratio greater than the stretching threshold as the image corresponding to the stretched label; and setting the processed image with the stretching ratio not greater than the stretching threshold as the image corresponding to the non-stretched label.

[0175] Specifically, the reference images can be selected from datasets such as AVA (Aesthetics Visual Analysis) and WATELOO, which are proportionally scaled datasets of images taken in natural environments. A uniform image processing type is randomly assigned to each of these reference images, categorizing them into three types: unstretched, height-stretched, and width-stretched. The unstretched reference images are not stretched, the height-stretched reference images are stretched vertically, and the width-stretched reference images are stretched horizontally. Then, a uniformly random stretching ratio (e.g., 0-1.2) is assigned to the height-stretched and width-stretched reference images; the stretching ratio for the unstretched reference images can be set to 0. Depending on the specific implementation, a stretching threshold (e.g., 0.6) is set. Images with a stretching ratio greater than the threshold are labeled as stretched, while images with a stretching ratio less than or equal to the threshold are labeled as unstretched. By setting different stretching ratios and processing types, selective stretching of the baseline image can generate diverse stretched and unstretched images, providing rich and varied training data for training models. This can improve the generalization performance of the model, enabling it to make accurate predictions when encountering new and unseen data.

[0176] Optionally, the images with known state labels can be uniformly processed images to meet the training data requirements of the image stretching recognition model and improve the training efficiency and effectiveness of the model. The processing includes: acquiring multiple original images with known state labels; processing the original images to obtain a first processed image; the first processed image being an image of part or all of the original images; scaling the first processed image proportionally and filling it into a preset-size template; if there are unfilled portions in the preset-size template, filling the unfilled portions with a preset number of pixels to obtain the image with known state labels.

[0177] Specifically, such as Figure 10As shown, the original image 1010 with known state labels is randomly cropped to obtain a first processed image 1020. This first processed image 1020 can be a sub-image of a region of the original image 1010, and the sub-image can be part or all of the image region of the original image 1010. Then, the first processed image 1020 is proportionally scaled and filled into a preset size template 1030 to unify the input training data to the same size, so that the image stretching recognition model does not need to pay attention to the image size during training. Since the aspect ratio of the first processed image 1020 may be different from that of the preset size template 1030, there may be an unfilled part 1040 in the preset size template after proportional scaling and filling. In order to ensure the integrity of the image, the unfilled part 1040 can be filled with preset pixels, such as RGB(128,128,128). Since the position and size of the region sub-images in the original image 1010 are random, the image stretching recognition model cannot simply determine whether the image is stretched by detecting the position of the filled region. Instead, it must learn deeper features related to image stretching. This way, the trained model can be more accurate and reliable, and can prevent the model from relying too much on the position of the filled region to determine whether the image is stretched.

[0178] Optionally, the first processed image is scaled proportionally and filled into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts, and a portion of the first processed image is covered with the preset pixels to obtain an image with a known state label.

[0179] Specifically, such as Figure 11 As shown, the image to be processed 1110 is an image that meets a preset size, including an image region 1111 and a pixel region 1112. The content of the image region 1111 is the first processed image described above; the content of the pixel region 1112 is the preset pixels filled during the proportional scaling and filling step of the first processed image. A portion of the image region 1111 (e.g., ...) is randomly selected... Figure 11 The central region (1120) is covered with the aforementioned preset pixels. This can, to some extent, simulate various situations that might occur in the real world. For example, some areas of an image may not provide useful information due to various reasons (such as occlusion, lighting changes, image quality, etc.). This allows the image stretching recognition model to be trained to correctly determine whether the image is stretched based on information from other areas even when information is missing or interfered with in certain areas, thus enhancing the model's robustness. Furthermore, this random coverage can further prevent the model from overly relying on a specific region of the image, such as excessively depending on the position and area of ​​the filled region to determine whether the image is stretched.

[0180] It should be noted that the preset pixels used for filling the image during preprocessing can be the same as the preset pixels mentioned above, so as to ensure that the model focuses more on the image itself rather than the pixel filling part, thereby improving the efficiency and accuracy of image stretching recognition.

[0181] S904: Input the image with known state label into the image stretching recognition model to obtain multiple different types of feature data, and output the predicted image stretching recognition result corresponding to the image with known state label based on the multiple different types of feature data.

[0182] Specifically, the image stretching recognition model includes a multi-branch deep learning network. Different branches can have different feature focuses and can extract different types of feature data. After inputting an image with known state labels (training data) into the image stretching recognition model, the model can extract multiple different types of feature data. Then, based on these feature data, subsequent feature recognition steps are performed to obtain the predicted image stretching recognition result.

[0183] S906, update the image stretching recognition model based on the predicted image stretching recognition results and status labels corresponding to the multiple images with known status labels.

[0184] Specifically, image stretching recognition models can be optimized using methods such as gradient descent. For example, the model is used to predict the image stretching recognition result on an image with known state labels (i.e., training data). The loss between the predicted image stretching recognition result and the actual state label is calculated. The gradient of the loss with respect to the model parameters is calculated using the backpropagation algorithm. The model parameters are updated according to the direction of the gradient to reduce the loss. The above steps are repeated until the model parameters reach a preset number of iterations or the model performance meets a preset standard.

[0185] It should be noted that the above image stretching recognition model is used to implement the image stretching recognition method described in the embodiments of this specification. The specific structure or configuration of this image stretching recognition model can be found above. Figure 2 The details of the image stretching recognition method will not be elaborated here.

[0186] This specification describes embodiments that acquire multiple images with known state labels, including stretched and unstretched labels. These images with known state labels are input into an image stretching recognition model to obtain multiple different types of feature data. Based on these different types of feature data, a predicted image stretching recognition result corresponding to the images with known state labels is output. The image stretching recognition model is updated based on the predicted image stretching recognition results and state labels corresponding to each of the multiple images with known state labels. Thus, by repeatedly comparing the model's prediction results with the actual labels and updating the model's parameters, the model's performance gradually improves. This allows the model to learn the key differences between stretched and unstretched images, thereby improving its accuracy and efficiency in recognizing stretched images on new images.

[0187] Please refer to the following. Figure 12 , Figure 12 An image stretching recognition device is provided as an exemplary embodiment of this specification.

[0188] like Figure 12 As shown, the image stretching recognition device 1200 includes:

[0189] The first acquisition module 1210 is used to acquire the image to be recognized;

[0190] The recognition module 1220 is used to input the image to be recognized into the image stretching recognition model to obtain multiple different types of feature data, and output the image stretching recognition result corresponding to the image to be recognized based on the multiple different types of feature data; the image stretching recognition model is trained based on multiple images with known state labels, and the state labels include stretching labels and non-stretching labels.

[0191] In one possible implementation, the image stretching recognition model described above includes a deep dual-branch convolutional neural network, which includes two different types of feature extraction branch networks.

[0192] The aforementioned identification module 1220 includes:

[0193] The first feature extraction unit is used to input the image to be identified into the first feature extraction branch network of the deep dual-branch convolutional neural network and output the first feature data corresponding to the image to be identified.

[0194] The second feature extraction unit is used to input the image to be identified into the second feature extraction branch network of the deep dual-branch convolutional neural network and output the second feature data corresponding to the image to be identified.

[0195] The fusion unit is used to fuse the first feature data and the second feature data to obtain fused feature data;

[0196] The output unit is used to output the image stretching recognition result corresponding to the image to be recognized based on the above-mentioned fused feature data.

[0197] In one possible implementation, both the first feature data and the second feature data mentioned above include a long dimension, a wide dimension, and a channel dimension.

[0198] The aforementioned fusion unit is specifically used for:

[0199] The first feature data and the second feature data are multiplied by a matrix to eliminate the long and wide dimensions of the first feature data and the second feature data, resulting in the fused feature data; the fused feature data includes the channel dimension.

[0200] In one possible implementation, the first feature extraction branch network of the aforementioned deep dual-branch convolutional neural network is an EfficientNet network, and the second feature extraction branch network of the aforementioned deep dual-branch convolutional neural network is a Spatial CNN network.

[0201] In one possible implementation, the image stretching recognition device 1200 further includes:

[0202] The generation module is used to rotate the above image to be identified to obtain multiple rotated images corresponding to the above image to be identified, and the rotation angles of the multiple rotated images are different.

[0203] The aforementioned identification module 1220 is specifically used for:

[0204] The rotated image corresponding to the above image to be identified is input into the image stretching recognition model to obtain multiple different types of feature data, and the image stretching recognition result corresponding to the above rotated image is output based on the multiple different types of feature data.

[0205] Based on the image stretching recognition results corresponding to each of the above-mentioned multiple rotated images, the image stretching recognition result of the above-mentioned image to be recognized is determined.

[0206] In one possible implementation, the image stretching recognition result includes the stretching probability;

[0207] The image stretching recognition device 1200 described above also includes a judgment module, used for:

[0208] Determine whether the above stretching probability is greater than the stretching determination threshold;

[0209] If so, then the image to be identified is determined to be a stretched image;

[0210] If not, then the image to be identified is determined to be a non-stretched image.

[0211] In one possible implementation, the image stretching recognition device 1200 further includes:

[0212] The preprocessing module is used to scale the image to be recognized proportionally and fill it into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts to obtain the image to be recognized that meets the preset size.

[0213] The division of modules in the above-described image stretching recognition device is for illustrative purposes only. In other embodiments, the image stretching recognition device can be divided into different modules as needed to complete all or part of the functions of the image stretching recognition device. The implementation of each module in the image stretching recognition device provided in the embodiments of this specification can be in the form of a computer program. This computer program can run on a terminal or server. The program modules constituted by this computer program can be stored in the memory of the terminal or server. When the computer program is executed by a processor, it implements all or part of the steps of the image stretching recognition method described in the embodiments of this specification.

[0214] Please refer to the following. Figure 13 , Figure 13 This specification provides an image stretching recognition model training apparatus as an exemplary embodiment. For example... Figure 13 As shown, the image stretching recognition model training device 1300 includes:

[0215] The second acquisition module 1310 is used to acquire multiple images with known state labels; the state labels include stretched labels and non-stretched labels.

[0216] The first training module 1320 is used to process the original image to obtain a first processed image; the first processed image is a region sub-image of the original image.

[0217] The second training module 1330 is used to update the image stretching recognition model based on the predicted image stretching recognition results and state labels corresponding to the multiple images with known state labels; the image stretching recognition model is used to implement the image stretching recognition method described in the embodiments of this specification.

[0218] In one possible implementation, the second acquisition module 1310 described above includes:

[0219] The first acquisition unit is used to acquire multiple reference images;

[0220] The type determination unit is used to determine the image processing type corresponding to each of the above multiple reference images; the above image processing types include non-stretching processing type, height stretching processing type and width stretching processing type.

[0221] The stretching processing unit is used to stretch the above reference image according to the corresponding image processing type and stretching ratio to obtain the processed image.

[0222] The first label unit is used to set the processed image with a stretching ratio greater than the stretching threshold as the image corresponding to the stretching label.

[0223] The second label unit is used to set the processed image with a stretching ratio not greater than the stretching threshold as the image corresponding to the non-stretched label.

[0224] In one possible implementation, the second acquisition module 1310 described above includes:

[0225] The second acquisition unit is used to acquire multiple original images with known state labels;

[0226] The first processing unit is used to process the original image to obtain a first processed image; the first processed image is an image of part or all of the original image.

[0227] The second processing unit is used to scale the first processed image proportionally and fill it into a preset size template. If there are unfilled parts in the preset size template, the preset pixels are filled into the unfilled parts to obtain an image with a known state label.

[0228] In one possible implementation, the second processing unit described above is specifically used for:

[0229] The first processed image is scaled proportionally and filled into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts, and a portion of the first processed image is covered with the preset pixels to obtain an image with a known state label.

[0230] The division of modules in the above-described image stretching recognition model training device is for illustrative purposes only. In other embodiments, the image stretching recognition model training device can be divided into different modules as needed to complete all or part of the functions of the above-described image stretching recognition model training device. The implementation of each module in the image stretching recognition model training device provided in the embodiments of this specification can be in the form of a computer program. This computer program can run on a terminal or server. The program modules constituted by this computer program can be stored in the memory of the terminal or server. When the computer program is executed by a processor, it implements all or part of the steps of the image stretching recognition model training method described in the embodiments of this specification.

[0231] Please see Figure 14 , Figure 14This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this specification. For example... Figure 14 As shown, the electronic device 1400 may include: at least one processor 1410, at least one communication bus 1420, a user interface 1430, at least one network interface 1440, and a memory 1450. The communication bus 1420 can be used to enable communication between the aforementioned components.

[0232] The user interface 1430 may include a display screen and a camera. Optional user interfaces may also include standard wired interfaces and wireless interfaces.

[0233] The network interface 1440 may optionally include a Bluetooth module, a Near Field Communication (NFC) module, a Wireless Fidelity (Wi-Fi) module, etc.

[0234] The processor 1410 may include one or more processing cores. The processor 1410 connects to various parts within the electronic device 1400 using various interfaces and lines. It executes various functions and processes data of the routing electronic device 1400 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1450, and by calling data stored in the memory 1450. Optionally, the processor 1410 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 1410 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 1410 and may be implemented as a separate chip.

[0235] The memory 1450 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 1450 may include a non-transitory computer-readable medium. The memory 1450 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1450 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a fetch function, a stretch recognition function, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 1450 may also be at least one storage device located remotely from the aforementioned processor 1410. Figure 14 As shown, the memory 1450, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and program instructions.

[0236] Specifically, the processor 1410 can be used to call program instructions stored in the memory 1450 and perform the following operations:

[0237] Acquire the image to be recognized;

[0238] The image to be identified is input into the image stretching recognition model to obtain multiple different types of feature data, and the image stretching recognition result corresponding to the image to be identified is output based on the multiple different types of feature data. The image stretching recognition model is trained on multiple images with known state labels, including stretching labels and non-stretching labels.

[0239] In some possible embodiments, the image stretching recognition model described above includes a deep dual-branch convolutional neural network, which includes two different types of feature extraction branch networks.

[0240] When the processor 1410 executes the process of inputting the image to be recognized into the image stretching recognition model, obtaining multiple different types of feature data, and outputting the image stretching recognition result corresponding to the image to be recognized based on the multiple different types of feature data, it specifically performs the following:

[0241] The image to be identified is input into the first feature extraction branch network of the deep dual-branch convolutional neural network, and the first feature data corresponding to the image to be identified is output.

[0242] The image to be identified is input into the second feature extraction branch network of the deep dual-branch convolutional neural network, and the second feature data corresponding to the image to be identified is output.

[0243] The first feature data and the second feature data are fused together to obtain fused feature data;

[0244] Based on the above fused feature data, the image stretching recognition result corresponding to the above image to be recognized is output.

[0245] In some possible embodiments, both the first feature data and the second feature data described above include a length dimension, a width dimension, and a channel dimension;

[0246] When the processor 1410 performs the fusion of the first feature data and the second feature data to obtain fused feature data, it specifically performs the following:

[0247] The first feature data and the second feature data are multiplied by a matrix to eliminate the long and wide dimensions of the first feature data and the second feature data, resulting in the fused feature data; the fused feature data includes the channel dimension.

[0248] In some possible embodiments, the first feature extraction branch network of the aforementioned deep dual-branch convolutional neural network is an EfficientNet network, and the second feature extraction branch network of the aforementioned deep dual-branch convolutional neural network is a SpatialCNN network.

[0249] In some possible embodiments, the processor 1410 described above is further configured to perform the following before executing the input image stretching recognition model of the image to be recognized:

[0250] The image to be identified is rotated to obtain multiple rotated images corresponding to the image to be identified. Each of the multiple rotated images has a different rotation angle.

[0251] The above-mentioned input of the image to be identified into the image stretching recognition model yields multiple different types of feature data, and outputs the image stretching recognition result corresponding to the image to be identified based on the multiple different types of feature data, including:

[0252] The rotated image corresponding to the above image to be identified is input into the image stretching recognition model to obtain multiple different types of feature data, and the image stretching recognition result corresponding to the above rotated image is output based on the multiple different types of feature data.

[0253] Based on the image stretching recognition results corresponding to each of the above-mentioned multiple rotated images, the image stretching recognition result of the above-mentioned image to be recognized is determined.

[0254] In some possible embodiments, the above image stretching recognition result includes stretching probability;

[0255] After the processor 1410 executes the process of inputting the image to be recognized into the image stretching recognition model, obtaining multiple different types of feature data, and outputting the image stretching recognition result corresponding to the image to be recognized based on the multiple different types of feature data, it is also used to execute:

[0256] Determine whether the above stretching probability is greater than the stretching determination threshold;

[0257] If so, then the image to be identified is determined to be a stretched image;

[0258] If not, then the image to be identified is determined to be a non-stretched image.

[0259] In some possible embodiments, after acquiring the image to be recognized and before inputting the image to be recognized into the image stretching recognition model, the processor 1410 is further configured to perform:

[0260] The image to be recognized is scaled proportionally and filled into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts to obtain the image to be recognized that meets the preset size.

[0261] In some possible embodiments, the aforementioned electronic device 1400 may be the aforementioned image stretching recognition model training device, and the aforementioned processor 1410 may further perform:

[0262] Acquire multiple images with known state labels, including stretched and unstretched labels;

[0263] The image with the known state label is input into the image stretching recognition model to obtain multiple different types of feature data, and the predicted image stretching recognition result corresponding to the image with the known state label is output based on the multiple different types of feature data.

[0264] The image stretching recognition model is updated based on the predicted image stretching recognition results and status labels corresponding to the multiple images with known status labels. The image stretching recognition model is used to implement the image stretching recognition method described in the embodiments of this specification.

[0265] In some possible embodiments, when the above-described electronic device 1400 acquires multiple images with known status tags, it specifically performs the following:

[0266] Acquire multiple baseline images;

[0267] Determine the image processing type corresponding to each of the above reference images; the above image processing types include non-stretching processing type, height stretching processing type, and width stretching processing type.

[0268] The above reference image is stretched according to the corresponding image processing type and stretching ratio to obtain the processed image;

[0269] The image with a stretching ratio greater than the stretching threshold is set as the image corresponding to the stretching label.

[0270] The processed image with a stretching ratio not greater than the stretching threshold is set as the image corresponding to the non-stretched label.

[0271] In some possible embodiments, when the above-described electronic device 1400 acquires multiple images with known status tags, it specifically performs the following:

[0272] Obtain multiple raw images with known state labels;

[0273] The original image is processed to obtain a first processed image; the first processed image is an image of part or all of the original image.

[0274] The first processed image is scaled proportionally and filled into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts to obtain an image with known status labels.

[0275] In some possible embodiments, when the electronic device 1400 performs the following operations: proportionally scaling the first processed image and filling it into a preset-size template, and if there are unfilled portions in the preset-size template, filling the unfilled portions with preset pixels to obtain an image with a known state label, the specific operations performed are as follows:

[0276] The first processed image is scaled proportionally and filled into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts, and a portion of the first processed image is covered with the preset pixels to obtain an image with a known state label.

[0277] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps in the above embodiments. If the constituent modules of the above-described image stretching recognition device or image stretching recognition model training device are implemented as software functional units and sold or used as independent products, they can be stored in the above-described computer-readable storage medium.

[0278] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0279] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation can be combined arbitrarily.

[0280] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made by those skilled in the art to the technical solutions of this specification without departing from the spirit of this specification should fall within the protection scope defined by the claims.

[0281] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims and specification may be performed in a different order than in the embodiments described in the specification and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for training an image stretching recognition model, the method comprising: Acquire multiple images with known state labels; the state labels include stretched labels and non-stretched labels; The image with the known state label is input into the image stretching recognition model to obtain multiple different types of feature data, and the predicted image stretching recognition result corresponding to the image with the known state label is output based on the multiple different types of feature data; The image stretching recognition model is updated based on the predicted image stretching recognition results and state labels corresponding to each of the multiple images with known state labels; The image stretching recognition model includes a deep dual-branch convolutional neural network, which includes two different types of feature extraction branch networks. The first feature extraction branch network of the deep dual-branch convolutional neural network is an EfficientNet network for extracting high-level global features, and the second feature extraction branch network of the deep dual-branch convolutional neural network is a Spatial CNN network for extracting spatial features. The image stretching recognition model is used for: Acquire the image to be recognized; The image to be identified is input into the first feature extraction branch network of the deep dual-branch convolutional neural network, and the first feature data corresponding to the image to be identified is output. The image to be identified is input into the second feature extraction branch network of the deep dual-branch convolutional neural network, and the second feature data corresponding to the image to be identified is output. The first feature data and the second feature data are fused to obtain fused feature data; Based on the fused feature data, the image stretching recognition result corresponding to the image to be recognized is output.

2. The method according to claim 1, wherein acquiring multiple images with known state labels includes: Acquire multiple baseline images; Determine the image processing type corresponding to each of the multiple reference images; The image processing types include non-stretch processing type, height stretch processing type, and width stretch processing type; The reference image is stretched according to the corresponding image processing type and stretching ratio to obtain the processed image; The image after processing, where the stretching ratio is greater than the stretching threshold, is set as the image corresponding to the stretching label. The processed image whose stretching ratio is not greater than the stretching threshold is set as the image corresponding to the non-stretched label.

3. The method according to claim 1, wherein acquiring multiple images with known state labels includes: Obtain multiple raw images with known state labels; The original image is processed to obtain a first processed image; The first processed image is an image of a portion or all of the original image; The first processed image is scaled proportionally and filled into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts to obtain an image with known status labels.

4. The method according to claim 3, wherein scaling the first processed image proportionally to fill a preset size template, and if there are unfilled portions in the preset size template, filling the unfilled portions with preset pixels to obtain an image with known state labels, comprises: The first processed image is scaled proportionally and filled into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts, and a portion of the first processed image is covered with the preset pixels to obtain an image with a known state label.

5. An image stretching recognition method, the method comprising: Acquire the image to be recognized; The image to be identified is input into the image stretching recognition model to obtain multiple different types of feature data, and the image stretching recognition result corresponding to the image to be identified is output based on the multiple different types of feature data; the image stretching recognition model is trained based on multiple images with known state labels, and the state labels include stretching labels and non-stretching labels; The image stretching recognition model is trained using the following method: multiple images with known state labels are acquired, including stretching labels and non-stretching labels; the images with known state labels are input into the image stretching recognition model to obtain multiple different types of feature data; the predicted image stretching recognition result corresponding to the image with known state labels is output based on the multiple different types of feature data; and the image stretching recognition model is updated based on the predicted image stretching recognition result and state label corresponding to each of the multiple images with known state labels. The image stretching recognition model includes a deep dual-branch convolutional neural network, which includes two different types of feature extraction branch networks. The first feature extraction branch network of the deep dual-branch convolutional neural network is an EfficientNet network for extracting high-level global features, and the second feature extraction branch network of the deep dual-branch convolutional neural network is a Spatial CNN network for extracting spatial features. The step of inputting the image to be identified into the image stretching recognition model to obtain multiple different types of feature data, and outputting the image stretching recognition result corresponding to the image to be identified based on the multiple different types of feature data, includes: The image to be identified is input into the first feature extraction branch network of the deep dual-branch convolutional neural network, and the first feature data corresponding to the image to be identified is output. The image to be identified is input into the second feature extraction branch network of the deep dual-branch convolutional neural network, and the second feature data corresponding to the image to be identified is output. The first feature data and the second feature data are fused to obtain fused feature data; Based on the fused feature data, the image stretching recognition result corresponding to the image to be recognized is output.

6. The method according to claim 5, wherein both the first feature data and the second feature data include a length dimension, a width dimension, and a channel dimension; The step of fusing the first feature data and the second feature data to obtain fused feature data includes: The first feature data and the second feature data are multiplied by a matrix to eliminate the long and wide dimensions of the first feature data and the second feature data, thereby obtaining the fused feature data; The fused feature data includes channel dimensions.

7. The method according to claim 5, wherein before inputting the image to be recognized into the image stretching recognition model, the method further comprises: The image to be identified is rotated to obtain multiple rotated images corresponding to the image to be identified, and each of the multiple rotated images corresponds to a different rotation angle; The step of inputting the image to be identified into the image stretching recognition model to obtain multiple different types of feature data, and outputting the image stretching recognition result corresponding to the image to be identified based on the multiple different types of feature data, includes: The rotated image corresponding to the image to be identified is input into the image stretching recognition model to obtain multiple different types of feature data, and the image stretching recognition result corresponding to the rotated image is output based on the multiple different types of feature data. Based on the image stretching recognition results corresponding to each of the multiple rotated images, the image stretching recognition result of the image to be recognized is determined.

8. The method according to claim 5, wherein the image stretching recognition result includes a stretching probability; After inputting the image to be identified into the image stretching recognition model to obtain multiple different types of feature data, and outputting the image stretching recognition result corresponding to the image to be identified based on the multiple different types of feature data, the method further includes: Determine whether the stretching probability is greater than the stretching determination threshold; If so, then the image to be identified is determined to be a stretched image; If not, then the image to be identified is determined to be an unstretched image.

9. The method according to claim 5, wherein after acquiring the image to be recognized and before inputting the image to be recognized into the image stretching recognition model, the method further comprises: The image to be recognized is scaled proportionally and filled into a preset size template. If there are unfilled parts in the preset size template, preset pixels are filled into the unfilled parts to obtain the image to be recognized that meets the preset size.

10. An image stretching recognition device, the device comprising: The first acquisition module is used to acquire the image to be recognized; The recognition module is used to input the image to be recognized into the image stretching recognition model to obtain multiple different types of feature data, and output the image stretching recognition result corresponding to the image to be recognized based on the multiple different types of feature data; the image stretching recognition model is trained based on multiple images with known state labels, and the state labels include stretching labels and non-stretching labels; The image stretching recognition model is trained using the following method: multiple images with known state labels are acquired, including stretching labels and non-stretching labels; the images with known state labels are input into the image stretching recognition model to obtain multiple different types of feature data; the predicted image stretching recognition result corresponding to the image with known state labels is output based on the multiple different types of feature data; and the image stretching recognition model is updated based on the predicted image stretching recognition result and state label corresponding to each of the multiple images with known state labels. The image stretching recognition model includes a deep dual-branch convolutional neural network, which includes two different types of feature extraction branch networks. The first feature extraction branch network of the deep dual-branch convolutional neural network is an EfficientNet network for extracting high-level global features, and the second feature extraction branch network of the deep dual-branch convolutional neural network is a Spatial CNN network for extracting spatial features. The identification module is specifically used for: The image to be identified is input into the first feature extraction branch network of the deep dual-branch convolutional neural network, and the first feature data corresponding to the image to be identified is output. The image to be identified is input into the second feature extraction branch network of the deep dual-branch convolutional neural network, and the second feature data corresponding to the image to be identified is output. The first feature data and the second feature data are fused to obtain fused feature data; Based on the fused feature data, the image stretching recognition result corresponding to the image to be recognized is output.

11. An image stretching recognition model training device, the device comprising: The second acquisition module is used to acquire multiple images with known state labels; the state labels include stretched labels and non-stretched labels. The first training module is used to input the image with the known state label into the image stretching recognition model to obtain multiple different types of feature data, and output the predicted image stretching recognition result corresponding to the image with the known state label based on the multiple different types of feature data; The second training module is used to update the image stretching recognition model based on the predicted image stretching recognition results and state labels corresponding to the multiple images with known state labels; The image stretching recognition model includes a deep dual-branch convolutional neural network, which includes two different types of feature extraction branch networks. The first feature extraction branch network of the deep dual-branch convolutional neural network is an EfficientNet network for extracting high-level global features, and the second feature extraction branch network of the deep dual-branch convolutional neural network is a Spatial CNN network for extracting spatial features. The image stretching recognition model is used for: Acquire the image to be recognized; The image to be identified is input into the first feature extraction branch network of the deep dual-branch convolutional neural network, and the first feature data corresponding to the image to be identified is output. The image to be identified is input into the second feature extraction branch network of the deep dual-branch convolutional neural network, and the second feature data corresponding to the image to be identified is output. The first feature data and the second feature data are fused to obtain fused feature data; Based on the fused feature data, the image stretching recognition result corresponding to the image to be recognized is output.

12. An electronic device, comprising: Processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the method as described in any one of claims 1-9.

13. A computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps of the method as claimed in any one of claims 1-9.

14. A computer program product comprising instructions that, when run on a computer or processor, cause the computer or processor to perform the method as described in any one of claims 1-9.

Images