Image detection method, storage medium and processor

Abstract

The present invention discloses an image detection method, storage medium, and processor. The method comprises: acquiring a target image; performing feature extraction on the target image to obtain multiple scale features, wherein each scale feature corresponds to a different scale; determining a target attention weight for the target scale feature based on a first scale feature among the multiple scale features, wherein the target scale feature is a scale feature other than the first scale feature among the multiple scale features; and detecting the target image based on the target attention weight and the target scale feature to obtain a detection result, wherein the detection result indicates whether a target object exists in the target image. The present invention solves the technical problem of low accuracy in image anomaly detection in related technologies.

Description

Technical Field

[0001] The present invention relates to the field of image detection, and in particular to an image detection method, a storage medium, and a processor. Background Art

[0002] Currently, product anomalies are common in industrial production lines, but manual inspection is tedious. Therefore, automating anomaly detection can significantly reduce labor costs and improve the detection rate of anomalies, thereby achieving continuous quality control and facilitating manual operations. However, current detection algorithms rely on the alignment of internal image contours. Large deviations in the image can significantly affect the detection results, resulting in low detection accuracy.

[0003] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0004] Embodiments of the present invention provide an image detection method, a storage medium, and a processor to at least solve the technical problem in the related art of low accuracy in detecting anomalies in images.

[0005] According to one aspect of an embodiment of the present invention, an image detection method is provided, comprising: acquiring a target image; performing feature extraction on the target image to obtain multiple scale features, wherein each scale feature corresponds to a different scale; determining a target attention weight of the target scale feature based on a first scale feature among the multiple scale features, wherein the target scale feature is a scale feature other than the first scale feature among the multiple scale features; and detecting the target image based on the target attention weight and the target scale feature to obtain a detection result, wherein the detection result is used to indicate whether a target object exists in the target image.

[0006] According to another aspect of an embodiment of the present invention, an image detection method is also provided, including: displaying a target image on an interactive interface; and displaying a detection result corresponding to the target image on the interactive interface when a target touch operation is sensed in the interactive interface, wherein the detection result is obtained by detecting the target image using a target attention weight and a target scale feature, and the detection result is used to indicate whether a target object exists in the target image, the target attention weight is determined based on a first scale feature among multiple scale features, and the target scale feature is a scale feature other than the first scale feature among the multiple scale features; the multiple scale features are obtained by performing feature extraction on the target image, and each scale feature corresponds to a different scale.

[0007] According to one aspect of an embodiment of the present invention, an image detection apparatus is provided, comprising: an acquisition module for acquiring a target image; an extraction module for performing feature extraction on the target image to obtain multiple scale features, wherein each scale feature corresponds to a different scale; a processing module for determining a target attention weight of a target scale feature based on a first scale feature among the multiple scale features, wherein the target scale feature is a scale feature other than the first scale feature among the multiple scale features; and a detection module for performing detection on a target image based on the target attention weight and the target scale feature to obtain a detection result, wherein the detection result is used to indicate whether a target object exists in the target image.

[0008] According to another aspect of an embodiment of the present invention, an image rendering method is also provided, including: displaying a target image on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device; performing feature extraction on the target image to obtain multiple scale features, wherein each scale feature corresponds to a different scale; determining a target attention weight of the target scale feature based on a first scale feature among the multiple scale features, wherein the target scale feature is a scale feature other than the first scale feature among the multiple scale features; detecting the target image based on the target attention weight and the target scale feature to obtain a detection result, wherein the detection result is used to indicate whether a target object exists in the target image; and driving the VR device or AR device to display the detection result.

[0009] According to another aspect of an embodiment of the present invention, a storage medium is further provided, which includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute any one of the above-mentioned image detection methods or an image rendering method.

[0010] According to another aspect of an embodiment of the present invention, a computer terminal is provided, comprising: a processor and a memory, wherein the processor is configured to run a program stored in the memory, wherein when the program is run, any one of the above-mentioned image detection methods or an image rendering method is executed.

[0011] In an embodiment of the present invention, a target image can be first acquired, and features can be extracted from the target image to obtain multiple scale features, wherein each scale feature corresponds to a different scale. Then, based on a first scale feature among the multiple scale features, a target attention weight of the target scale feature is determined, wherein the target scale feature is a scale feature other than the first scale feature among the multiple scale features. Finally, the target image is detected based on the target attention weight and the target scale feature to obtain a detection result, wherein the detection result is used to indicate whether a target object exists in the target image, thereby improving the accuracy of detecting the target object in the target image. Through the attention mechanism, the first scale feature can be used to query the feature information of the target scale feature to obtain the target attention weight corresponding to the target scale feature. By fusing the target scale features through the target attention weight, the feature information contained in the multiple scale features can be retained, thereby improving the accuracy of target image detection, thereby solving the technical problem of low accuracy of image anomaly detection in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0013] Figure 1 This is a hardware structure block diagram of a computer terminal (or mobile device) for a real image detection method according to an embodiment of the present application;

[0014] Figure 2 is a flow chart of the image detection method according to the first embodiment of the present application;

[0015] Figure 3 is a structural diagram of an image detection method according to an embodiment of the present application;

[0016] Figure 4 is a structural diagram of another image detection method according to an embodiment of the present application;

[0017] Figure 5 is a flow chart of an image detection method according to the second embodiment of the present application;

[0018] Figure 6 is a schematic diagram of an image detection device according to the third embodiment of the present application;

[0019] Figure 7 is a schematic diagram of an image detection device according to the fourth embodiment of the present application;

[0020] Figure 8 It is a block diagram of a computer terminal according to an embodiment of the present application;

[0021] Figure 9 is a flowchart of an image rendering method according to an embodiment of the present application;

[0022] Figure 10 A schematic diagram of a hardware environment for implementing an image rendering method according to an embodiment of the present application;

[0023] Figure 11 This is another schematic diagram of a hardware environment for implementing an image rendering method according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following interpretations:

[0027] Small objects: In object detection applications, there's no strict or clear definition of small objects. Generally speaking, a small object refers to an object whose bounding box is smaller than 1 / 50 of the image's width or height (in pixels). For example, if an image is 1000x1000 and its bounding box is smaller than 20x20, it's considered a small object.

[0028] Currently, among many abnormal target detection algorithms, the PaDiM algorithm (anomaly detection algorithm) performs better, but it relies more on alignment with the internal contours of the image. If there is a large deviation in the image, it will have a greater impact on the detection results. Adding an alignment model will reduce the speed of the model. In addition, the PaDiM algorithm has a high small target missed detection rate. If only the underlying features are used, the overall detection performance will be reduced.

[0029] The embodiment of the present application provides an image detection method, which constructs a framework for an abnormal target detection method based on a conditional flow model of an attention mechanism, which can meet practical application scenarios, especially scenarios involving small targets, and the method has lower requirements for internal contour alignment of the image.

[0030] Example 1

[0031] According to an embodiment of the present invention, an embodiment of an image detection method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG1 shows a hardware structure block diagram of a computer terminal (or mobile device) for a real image detection method. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more processors (illustrated as 102a, 102b, ..., 102n in the figure) (the processor may include but is not limited to a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. Those skilled in the art will understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0033] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." This data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be fully or partially integrated into any of the other components of the computer terminal 10 (or mobile device). As discussed in the embodiments of this application, this data processing circuitry functions as a processor control (e.g., selecting a variable resistor terminal path connected to an interface).

[0034] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the image detection method in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the above-mentioned image detection method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0035] Transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of computer terminal 10. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module configured to communicate with the Internet wirelessly.

[0036] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0037] It should be noted that, in some optional embodiments, the above Figure 1 The computer device (or mobile device) shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the aforementioned computer device (or mobile device).

[0038] Under the above operating environment, this application provides Figure 2 The image detection method shown. Figure 2 FIG. 4 is a flowchart of an image detection method according to a first embodiment of the present invention.

[0039] Step S202: Acquire a target image.

[0040] The target image mentioned above may be an image of a product to be inspected.

[0041] In an optional embodiment, the target image can be obtained by photographing the product using a photographing device, wherein the photographing device includes but is not limited to a camera, a video camera, an infrared camera, a video camera, etc. The photographing device can be installed in an industrial production line, and the target image can be obtained by photographing the product to be detected for abnormalities using the photographing device.

[0042] In another optional embodiment, the product image to be inspected may also be acquired from an image library or other channels.

[0043] Step S204: extract features from the target image to obtain multiple scale features.

[0044] Among them, each scale feature corresponds to a different scale.

[0045] Each of the multiple scale features corresponds to a different scale, that is, the resolution corresponding to the layer where each scale feature is located is different.

[0046] In an optional embodiment, feature extraction can be performed on the target image through an encoder to obtain multiple scale features, wherein the encoder can be a convolutional neural network feature extractor with multi-scale pyramid pooling. It should be noted that the pyramid pooling layer can capture global and local semantic information and the receptive field can continuously grow from top to bottom.

[0047] Step S206 : determining a target attention weight of a target scale feature based on a first scale feature among the multiple scale features.

[0048] The target scale feature is a scale feature other than the first scale feature among the multiple scale features.

[0049] The first scale feature mentioned above may be a characteristic scale in an intermediate layer among the multiple scale features.

[0050] The target scale feature may include at least one of the following: a high-level scale feature and a low-level scale feature. The target scale feature may also be a scale feature of any layer among multiple scale features. The high-level scale feature may be a high-level semantic feature, and the low-level scale feature may be a low-level texture feature.

[0051] The above-mentioned target attention weight may include at least one of the following: a high-level attention weight corresponding to a high-level scale feature, and a low-level attention weight corresponding to a low-level scale feature.

[0052] In an optional embodiment, a multi-head attention mechanism can be introduced, and the intermediate layer features are used as query vectors, which are multiplied with the high-level scale features and the low-level scale features to obtain high-level attention weights and low-level attention weights, respectively. This can enhance the expressiveness of the low-level texture features and improve the detection rate of small targets. It can also retain the expressiveness of the high-level semantic features and stabilize the overall detection performance of the model.

[0053] In order to utilize high-level features and low-level features through data adjustment, the hyperparameter W can be designed for the attention weights corresponding to high-level scale features and low-level scale features respectively. H and W L , in order to facilitate the weighting adjustment of features of different scales.

[0054] In another optional embodiment, after determining the target scale feature and the target attention weight, the method further includes: obtaining a target hyperparameter corresponding to the target scale feature, adjusting the target attention weight based on the target hyperparameter to obtain an adjusted target attention weight, and detecting the target image based on the adjusted target attention weight and the target scale feature to obtain a detection result, so as to more accurately detect abnormalities in the target image.

[0055] Step S208: Detect the target image based on the target attention weight and the target scale feature to obtain a detection result.

[0056] The detection result is used to indicate whether the target object exists in the target image.

[0057] The target object mentioned above may be an abnormal object, that is, a part that should not exist in the target image.

[0058] In an optional embodiment, the target attention weight can be fused with the corresponding target scale feature to obtain a feature vector corresponding to the target scale feature. The feature vector is used as the input of a subsequent conditional normalization flow model to obtain a target likelihood value corresponding to the feature vector. The log-likelihood score corresponding to the target scale feature can be determined based on the target likelihood value. An anomaly score map can be generated based on the log-likelihood score. The anomaly score map can be used to locate abnormal objects in the target image. By detecting the anomaly score map, it can be determined whether abnormal objects appear in the target image.

[0059] Through the above steps, first, a target image can be obtained, and features of the target image can be extracted to obtain multiple scale features, wherein each scale feature corresponds to a different scale. Then, based on the first scale feature among the multiple scale features, a target attention weight of the target scale feature is determined, wherein the target scale feature is a scale feature other than the first scale feature among the multiple scale features. Finally, the target image is detected based on the target attention weight and the target scale feature to obtain a detection result, wherein the detection result is used to indicate whether the target object exists in the target image, thereby improving the accuracy of detecting the target object in the target image. Through the attention mechanism, the first scale feature can be used to query the feature information of the target scale feature to obtain the target attention weight corresponding to the target scale feature. By fusing the target scale feature through the target attention weight, the feature information contained in the multiple scale features can be retained, thereby improving the accuracy of target image detection, thereby solving the technical problem of low accuracy of image anomaly detection in the related art.

[0060] In the above embodiment of the present application, determining the target attention weight of the target scale feature based on the first scale feature among the multiple scale features includes: obtaining the target attention weight based on the product of the first scale feature and the target scale feature.

[0061] In an optional embodiment, when the target scale feature includes a high-level scale feature and a low-level scale feature, the attention weight corresponding to the high-level scale feature can be obtained according to the product of the first scale feature and the high-level scale feature; and the attention weight corresponding to the low-level scale feature can be obtained according to the product of the first scale feature and the low-level scale feature.

[0062] In the above embodiment of the present application, the target image is detected based on the target attention weight and the target scale feature to obtain the detection result, including: generating a target feature vector based on the product of the target attention weight and the target scale feature; and detecting the target image based on the target feature vector to obtain the detection result.

[0063] In an optional embodiment, when the target attention weight includes the attention weight corresponding to the high-level scale feature and the attention weight corresponding to the low-level scale feature, the feature vector corresponding to the high-level scale feature can be generated by fusion based on the product of the attention weight corresponding to the high-level scale feature and the high-level scale feature; the feature vector corresponding to the low-level scale feature can be generated by fusion based on the product of the attention weight corresponding to the low-level scale feature and the low-level scale feature; and the target image can be detected based on the obtained feature vectors corresponding to the two different scales to obtain a detection result with higher accuracy.

[0064] In the above embodiment of the present application, the target image is detected based on the target feature vector to obtain the detection result, including: using the target flow model to perform likelihood estimation on the target feature vector to generate a target likelihood value corresponding to the target feature vector; obtaining a likelihood score map corresponding to the target likelihood value, wherein the likelihood score map is used to represent the resolution of features at different scales; and detecting the target image based on the likelihood score map to obtain the detection result.

[0065] The target flow model mentioned above may be a conditional normalized flow model.

[0066] The target feature vector mentioned above has position encoding information. The target feature vector may include feature vectors corresponding to high-level scale features, and may also include feature vectors corresponding to low-level scale features. The target feature vector may also include feature vectors corresponding to high-level scale features and feature vectors corresponding to low-level scale features.

[0067] In an optional embodiment, a target flow model can be used to perform multi-scale likelihood estimation on multiple feature vectors with position encoding information to generate a likelihood value corresponding to each feature vector. A likelihood score map can be obtained for the likelihood value corresponding to each feature vector, so that the resolution of the scale feature corresponding to the feature vector can be determined based on the likelihood score map. The likelihood score map can also be used to locate the abnormal area in the target image, so that the abnormal object in the target image can be detected.

[0068] In the above embodiment of the present application, the target image is detected based on the likelihood score map to obtain a detection result, including: processing the likelihood score map to generate a target score map, wherein the scale corresponding to the target score map is the same as that of the target image; and detecting the target image based on the target score map to obtain a detection result.

[0069] In an optional embodiment, the likelihood score map can be normalized to convert the values ​​in the likelihood score map to between [0, 1]. The converted likelihood score map is then upsampled to the scale of the target image using a bilinear interpolation method and then normalized to obtain a target score map. Since the target score image has the same scale as the target image, a more accurate detection result can be obtained by detecting the target image using the target score map.

[0070] In the above embodiment of the present application, the target image is detected based on the target score map to obtain the detection result, including: processing the target score map based on a preset pixel threshold to obtain a target mask map corresponding to the target score map; and detecting the target mask map to obtain the detection result.

[0071] The preset pixel threshold may be pre-set or may be a threshold at which the index score reaches a maximum value when the target flow model is trained using a test set used in the training process.

[0072] In an optional embodiment, the target score map can be converted into a target mask map based on a preset pixel threshold, so that the converted target mask map can be used to locate the target object with an anomaly. When the score in the target score map is greater than the preset pixel threshold, it indicates that the pixel point has an anomaly and is represented by 1 in the target mask map. When the score in the target score map is less than the preset pixel threshold, it indicates that the pixel point has no anomaly and is represented by 0 in the target mask map.

[0073] In the above embodiment of the present application, obtaining a likelihood score map corresponding to a target likelihood value includes: determining a log-likelihood score corresponding to the target likelihood value based on a log-likelihood formula; and determining a likelihood score map corresponding to the log-likelihood score.

[0074] The above log-likelihood formula is as follows:

[0075] ;

[0076] in, is the log-likelihood score, are the parameters of the target flow model, is the feature vector (high-level feature vector or low-level feature vector) reconstructed by the target flow model, Encode information for position; is the eigenvector corresponding to the target likelihood value, is the Jacobian matrix of the multivariable substitution function in the target flow model, and D is the divergence.

[0077] In an optional embodiment, the above-mentioned log-likelihood formula may be used to determine the log-likelihood score corresponding to the target likelihood value, which serves as a basis for generating a likelihood score map.

[0078] In the above embodiment of the present application, processing the likelihood score map to generate a target score map includes: normalizing the likelihood score map to generate a first score map corresponding to the target scale feature; upsampling the first score map corresponding to the target scale feature using a bilinear interpolation method to generate a second score map corresponding to the target scale feature, wherein the scale of the second score map is the same as the scale of the scale feature corresponding to the target score map; and normalizing the second score map to obtain the target score map.

[0079] The first score map mentioned above is a likelihood score map after normalization.

[0080] The target score map mentioned above can be the final anomaly score map.

[0081] In an optional embodiment, when the target scale features only include high-level scale features, the likelihood score maps corresponding to the high-level scale features can be normalized to obtain a first score map corresponding to the high-level scale features. The first score map corresponding to the high-level scale features can be upsampled using bilinear interpolation to generate a second score map corresponding to the high-level scale features. The second score map can be normalized to obtain a final score map corresponding to the high-level scale features. When the target scale features only include low-level scale features, the likelihood score maps corresponding to the low-level scale features can be normalized to obtain a first score map corresponding to the low-level scale features. The first score map corresponding to the low-level scale features can be upsampled using bilinear interpolation to generate a second score map corresponding to the low-level scale features. The second score map can be normalized to obtain a final score map corresponding to the low-level scale features.

[0082] In another optional embodiment, when the target scale feature includes a high-level scale feature and a low-level scale feature, a score map corresponding to the high-level scale feature and a score map corresponding to the low-level scale feature may be generated respectively, and the two score maps may be merged to generate the above-mentioned second score map.

[0083] Optionally, the likelihood score map corresponding to each scale feature can be used The likelihood score map is transformed to between [0, 1] to obtain the first score map mentioned above.

[0084] In the above embodiment of the present application, when the target scale feature includes multiple sub-features, upsampling the first score map corresponding to the target scale feature using a bilinear interpolation method to generate a second score map corresponding to the target scale feature includes: upsampling the multiple sub-features using a bilinear interpolation method to generate multiple third score maps corresponding to the multiple sub-features; and merging the multiple third score maps to generate the second score map.

[0085] The aforementioned multiple sub-features may be high-level scale features and low-level scale features.

[0086] In an optional embodiment, a bilinear interpolation method may be used to upsample the multiple sub-features to generate multiple third score maps corresponding to the multiple sub-features. The multiple third score maps may be added to obtain the second score map.

[0087] In the above embodiment of the present application, the method also includes: obtaining a sample feature vector and a sample likelihood value corresponding to the sample feature vector; using a target flow model to estimate the sample feature vector and generate a target likelihood value corresponding to the sample feature vector; obtaining a first distribution corresponding to the sample likelihood value and a second distribution corresponding to the target likelihood value; constructing a target loss function based on the first distribution and the second distribution; and using the target loss function to update the target flow model to obtain an updated result.

[0088] The above-mentioned sample feature vector and sample likelihood value may be preset.

[0089] The first distribution can be a predicted distribution that is reversibly reconstructed through a target flow model to fit a true normal distribution, and the second distribution can minimize the true distribution.

[0090] In an optional embodiment, a sample feature vector and a sample likelihood value corresponding to the sample feature vector can be obtained, and the sample feature vector can be estimated using a target flow model to obtain a target likelihood value corresponding to the sample feature vector. The predicted distribution corresponding to the sample likelihood value and the minimized distribution corresponding to the target likelihood value can be obtained as the KL divergence to construct a target loss function. The target flow model can be updated according to the target loss function to obtain an updated result.

[0091] The above objective loss function can be expressed by the following formula:

[0092] ;

[0093] in, is the KL divergence, is the feature vector reconstructed by the target flow model, c is the position coding information, N is the number of sample feature vectors, is a constant, are the parameters of the target flow model; is the eigenvector corresponding to the target likelihood value, is the Jacobian matrix of the multivariable substitution function in the target flow model.

[0094] The conditional normalized flow model is used to learn and reconstruct the global features of normal patterns in the image from multiple scales. This ensures that even when the contour position deviation in the image is large, the model can still improve the difference between normal and abnormal areas based on global features, and can reduce the model's demand for image alignment.

[0095] Figure 3 This is a structural diagram of an image detection method according to an embodiment of the present application, wherein a target image can be input into an encoder to obtain multi-scale features of the target image, and the multi-scale features are fused through a feature fusion module to obtain high-level scale features and low-level scale features. The high-level scale features and low-level scale features are processed using a conditional normalized flow model to obtain likelihood score maps corresponding to the high-level scale features and likelihood score maps corresponding to the low-level scale features. The likelihood score maps can be processed using a multi-scale aggregation module to obtain a final target score map. The target image is detected based on the target score map to obtain a detection result. The abnormal object in the detection result can be part of the final dark area on the way.

[0096] Figure 4 This is a structural diagram of another image detection method according to an embodiment of the present application. The target image can be input into the encoder to obtain the multi-scale features of the target image. After extracting the multi-scale features of the image, a multi-head attention mechanism is introduced. The intermediate layer scale features are used as query vectors and multiplied with the high-level scale features and the low-level scale features to obtain the attention weights corresponding to the high-level scale features and the attention weights corresponding to the low-level scale features. Then, in order to adjust the utilization of high-level features and low-level features for different data, a hyperparameter W for the attention weights of the two feature layers can be designed. H and W L , in order to facilitate the weighting adjustment of features at different scales, the two weighted attention weights are multiplied by the features at their respective scales to obtain the fused feature vectors of the two scales, and the feature vectors are used as the input of the subsequent conditional normalization flow model.

[0097] Through the above content, we can provide a self-supervised method for abnormal target detection in industrial vision. After extracting multi-scale features through a convolutional neural network, a conditional normalized flow model is used to learn and reconstruct the global features of normal patterns in the image from multi-scale levels. At the same time, an attention mechanism is introduced to use intermediate layer features to query high-level semantic features and underlying texture features, so that the model can improve the detection rate of abnormal small targets while retaining the expressive power of high-level semantic features.

[0098] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0099] Through the description of the above embodiments, those skilled in the art will clearly understand that the methods according to the above embodiments can be implemented using software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is the more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk) and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0100] Example 2

[0101] According to an embodiment of the present application, an image detection method embodiment is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0102] Figure 5 is a flow chart of an image detection method according to the second embodiment of the present application. Figure 6 As shown, the method may include the following steps:

[0103] Step S502: displaying the target image on the interactive interface;

[0104] Step S504 : When a target touch operation is sensed in the interactive interface, a detection result corresponding to the target image is displayed in the interactive interface.

[0105] The target touch operation mentioned above may be an operation in which the user touches the interactive interface, wherein the target touch operation may be used to detect the target image, obtain the detection result, and display the detection result in the interactive interface for the user to view.

[0106] In an optional embodiment, when viewing the test results on the interactive interface, the user can modify the test results to improve the accuracy of the test results.

[0107] Among them, the detection result is obtained by detecting the target image through the target attention weight and the target scale feature. The detection result is used to indicate whether the target object exists in the target image. The target attention weight is determined based on the first scale feature among multiple scale features. The target scale feature is the scale feature other than the first scale feature among the multiple scale features; the multiple scale features are obtained by feature extraction of the target image, and each scale feature corresponds to a different scale.

[0108] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0109] Example 3

[0110] According to an embodiment of the present invention, an image detection device for implementing the above-mentioned image detection method is also provided. Figure 6 is a schematic diagram of an image detection device according to the third embodiment of the present application, such as Figure 6 As shown, the device includes: an acquisition module 602 , an extraction module 604 , a determination module 606 , and a detection module 608 .

[0111] Among them, the acquisition module is used to acquire the target image; the extraction module is used to extract features of the target image to obtain multiple scale features, wherein each scale feature corresponds to a different scale; the determination module is used to determine the target attention weight of the target scale feature based on the first scale feature among the multiple scale features, wherein the target scale feature is the scale feature other than the first scale feature among the multiple scale features; the detection module is used to detect the target image based on the target attention weight and the target scale feature to obtain a detection result, wherein the detection result is used to indicate whether the target object exists in the target image.

[0112] It should be noted that the acquisition module 602, extraction module 604, determination module 606, and detection module 608 correspond to steps S202 to S208 in Example 1. The examples and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0113] In the embodiment of the present application, the determination module is further configured to obtain a target attention weight based on the product of the first scale feature and the target scale feature.

[0114] In the embodiment of the present application, the detection module includes: a generation unit and a detection unit.

[0115] Among them, the generation unit is used to generate a target feature vector based on the product of the target attention weight and the target scale feature; the detection unit is used to detect the target image based on the target feature vector to obtain a detection result.

[0116] In an embodiment of the present application, the detection unit is also used to use the target flow model to perform likelihood estimation on the target feature vector to generate a target likelihood value corresponding to the target feature vector; the detection unit is also used to obtain a likelihood score map corresponding to the target likelihood value, wherein the likelihood score map is used to represent the resolution of features at different scales; the detection unit is also used to detect the target image based on the likelihood score map to obtain a detection result.

[0117] In an embodiment of the present application, the detection unit is further used to process the likelihood score map to generate a target score map, wherein the scale corresponding to the target score map is the same as that of the target image; the detection unit is further used to detect the target image based on the target score map to obtain a detection result.

[0118] In an embodiment of the present application, the detection unit is further used to process the target score map based on a preset pixel threshold to obtain a target mask map corresponding to the target score map; the detection unit is further used to detect the target mask map to obtain a detection result.

[0119] In an embodiment of the present application, the acquisition module is further used to determine the log-likelihood score corresponding to the target likelihood value based on the log-likelihood formula; the acquisition module is further used to determine the likelihood score map corresponding to the log-likelihood score.

[0120] In an embodiment of the present application, the detection unit is further used to normalize the likelihood score map to generate a first score map corresponding to the target scale feature; the detection unit is further used to upsample the first score map corresponding to the target scale feature using a bilinear interpolation method to generate a second score map corresponding to the target scale feature, wherein the scale of the second score map is the same as the scale of the scale feature corresponding to the target score map; the detection unit is further used to normalize the second score map to obtain the target score map.

[0121] In an embodiment of the present application, the detection unit is further used to upsample the multiple sub-features using a bilinear interpolation method to generate multiple third score maps corresponding to the multiple sub-features; the detection unit is also used to merge the multiple third score maps to generate a second score map.

[0122] In an embodiment of the present application, the device further includes: an estimation module, a construction module, and an update module.

[0123] Among them, the acquisition module is also used to obtain the sample feature vector and the sample likelihood value corresponding to the sample feature vector; the estimation module is used to use the target flow model to estimate the sample feature vector and generate the target likelihood value corresponding to the sample feature vector; the acquisition module is also used to obtain the first distribution corresponding to the sample likelihood value and the second distribution corresponding to the target likelihood value; the construction module is used to construct the target loss function based on the first distribution and the second distribution; the update module is used to update the target flow model using the target loss function to obtain the updated result.

[0124] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0125] Example 4

[0126] According to an embodiment of the present invention, an image detection device for implementing the above-mentioned image detection method is also provided. Figure 7 is a schematic diagram of an image detection device according to the fourth embodiment of the present application; Figure 7 As shown, the device includes: a display module 702 and a sensing module 704.

[0127] Among them, the display module is used to display the target image on the interactive interface; the sensing module is used to display the detection result corresponding to the target image on the interactive interface when a target touch operation is sensed in the interactive interface, wherein the detection result is obtained by detecting the target image through the target attention weight and the target scale feature, and the detection result is used to indicate whether the target object exists in the target image, and the target attention weight is determined based on the first scale feature among multiple scale features, and the target scale feature is the scale feature of the multiple scale features other than the first scale feature; the multiple scale features are obtained by feature extraction of the target image, and each scale feature corresponds to a different scale.

[0128] It should be noted that the display module 702 and the sensing module 704 correspond to steps S502 to S504 in Example 1. The examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0129] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0130] Example 5

[0131] The embodiment of the present invention can provide a computer terminal, which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal can also be replaced by a terminal device such as a mobile terminal.

[0132] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0133] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the image detection method: obtaining a target image; performing feature extraction on the target image to obtain multiple scale features, wherein each scale feature corresponds to a different scale; determining a target attention weight of the target scale feature based on a first scale feature among the multiple scale features, wherein the target scale feature is a scale feature other than the first scale feature among the multiple scale features; detecting the target image based on the target attention weight and the target scale feature to obtain a detection result, wherein the detection result is used to indicate whether a target object exists in the target image.

[0134] Optionally, Figure 8 This is a structural block diagram of a computer terminal according to an embodiment of the present application. Figure 8 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors and a memory.

[0135] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image detection method and device in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image detection method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to terminal A via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0136] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain a target image; perform feature extraction on the target image to obtain multiple scale features, wherein each scale feature corresponds to a different scale; determine a target attention weight of the target scale feature based on a first scale feature among the multiple scale features, wherein the target scale feature is a scale feature other than the first scale feature among the multiple scale features; detect the target image based on the target attention weight and the target scale feature to obtain a detection result, wherein the detection result is used to indicate whether a target object exists in the target image.

[0137] Optionally, the processor may further execute a program code of the following steps: obtaining a target attention weight based on a product of the first scale feature and the target scale feature.

[0138] Optionally, the processor may further execute program code of the following steps: generating a target feature vector based on the product of the target attention weight and the target scale feature; and detecting the target image based on the target feature vector to obtain a detection result.

[0139] Optionally, the processor may also execute the program code of the following steps: using the target flow model to perform likelihood estimation on the target feature vector to generate a target likelihood value corresponding to the target feature vector; obtaining a likelihood score map corresponding to the target likelihood value, wherein the likelihood score map is used to represent the resolution of features at different scales; and detecting the target image based on the likelihood score map to obtain a detection result.

[0140] Optionally, the processor may further execute program code of the following steps: processing the likelihood score map to generate a target score map, wherein the scale corresponding to the target score map is the same as that of the target image; and detecting the target image based on the target score map to obtain a detection result.

[0141] Optionally, the processor may further execute program code of the following steps: processing the target score map based on a preset pixel threshold to obtain a target mask map corresponding to the target score map; and detecting the target mask map to obtain a detection result.

[0142] Optionally, the processor may further execute program code for the following steps: determining a log-likelihood score corresponding to the target likelihood value based on a log-likelihood formula; and determining a likelihood score map corresponding to the log-likelihood score.

[0143] Optionally, the processor may further execute program code of the following steps: normalizing the likelihood score map to generate a first score map corresponding to the target scale feature; upsampling the first score map corresponding to the target scale feature using a bilinear interpolation method to generate a second score map corresponding to the target scale feature, wherein the scale of the second score map is the same as the scale of the scale feature corresponding to the target score map; and normalizing the second score map to obtain the target score map.

[0144] Optionally, the processor may further execute program code of the following steps: when the target scale feature includes multiple sub-features, upsampling the multiple sub-features using bilinear interpolation to generate multiple third score maps corresponding to the multiple sub-features; and merging the multiple third score maps to generate a second score map.

[0145] Optionally, the processor may also execute the program code of the following steps: obtaining a sample feature vector and a sample likelihood value corresponding to the sample feature vector; estimating the sample feature vector using a target flow model to generate a target likelihood value corresponding to the sample feature vector; obtaining a first distribution corresponding to the sample likelihood value and a second distribution corresponding to the target likelihood value; constructing a target loss function based on the first distribution and the second distribution; and updating the target flow model using the target loss function to obtain an updated result.

[0146] The processor can call information and applications stored in the memory through the transmission device to perform the following steps: displaying a target image on an interactive interface; when a target touch operation is sensed in the interactive interface, displaying a detection result corresponding to the target image in the interactive interface, wherein the detection result is obtained by detecting the target image using a target attention weight and a target scale feature, and the detection result is used to indicate whether a target object exists in the target image, the target attention weight is determined based on a first scale feature among multiple scale features, and the target scale feature is a scale feature other than the first scale feature among the multiple scale features; the multiple scale features are obtained by extracting features from the target image, and each scale feature corresponds to a different scale.

[0147] An embodiment of the present invention provides an image detection method. First, a target image can be acquired, and features of the target image can be extracted to obtain multiple scale features, wherein each scale feature corresponds to a different scale. Then, based on a first scale feature among the multiple scale features, a target attention weight of the target scale feature is determined, wherein the target scale feature is a scale feature other than the first scale feature among the multiple scale features. Finally, the target image is detected based on the target attention weight and the target scale feature to obtain a detection result. The detection result is used to indicate whether a target object exists in the target image, thereby improving the accuracy of detecting the target object in the target image. Through the attention mechanism, the first scale feature can be used to query feature information of the target scale feature to obtain the target attention weight corresponding to the target scale feature. The target scale features are fused using the target attention weight, and the feature information contained in the multiple scale features can be retained, thereby improving the accuracy of target image detection, thereby solving the technical problem of low accuracy of image anomaly detection in the related art.

[0148] It can be understood by those skilled in the art that Figure 8 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 8 It does not limit the structure of the above electronic device. For example, the computer terminal 10 may also include Figure 8 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 8 Different configurations shown.

[0149] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0150] Example 6

[0151] The embodiment of the present invention further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the image detection method provided in the first embodiment.

[0152] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0153] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: acquiring a target image; performing feature extraction on the target image to obtain multiple scale features, wherein each scale feature corresponds to a different scale; determining a target attention weight of the target scale feature based on a first scale feature among the multiple scale features, wherein the target scale feature is a scale feature among the multiple scale features other than the first scale feature; detecting the target image based on the target attention weight and the target scale feature to obtain a detection result, wherein the detection result is used to indicate whether a target object exists in the target image.

[0154] Optionally, the storage medium is further configured to store a program code for executing the following steps: obtaining a target attention weight based on a product of the first scale feature and the target scale feature.

[0155] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: generating a target feature vector based on the product of the target attention weight and the target scale feature; detecting the target image based on the target feature vector to obtain a detection result.

[0156] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: using the target flow model to perform likelihood estimation on the target feature vector to generate a target likelihood value corresponding to the target feature vector; obtaining a likelihood score map corresponding to the target likelihood value, wherein the likelihood score map is used to represent the resolution of features at different scales; and detecting the target image based on the likelihood score map to obtain a detection result.

[0157] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: processing the likelihood score map to generate a target score map, wherein the scale corresponding to the target score map is the same as that of the target image; and detecting the target image based on the target score map to obtain a detection result.

[0158] Optionally, the above-mentioned storage medium is further configured to store program code for executing the following steps: processing the target score map based on a preset pixel threshold to obtain a target mask map corresponding to the target score map; and detecting the target mask map to obtain a detection result.

[0159] Optionally, the storage medium is further configured to store program codes for executing the following steps: determining a log-likelihood score corresponding to the target likelihood value based on a log-likelihood formula; and determining a likelihood score map corresponding to the log-likelihood score.

[0160] Optionally, the storage medium is further configured to store program code for executing the following steps: normalizing the likelihood score map to generate a first score map corresponding to the target scale feature; upsampling the first score map corresponding to the target scale feature using a bilinear interpolation method to generate a second score map corresponding to the target scale feature, wherein the scale of the second score map is the same as the scale of the scale feature corresponding to the target score map; and normalizing the second score map to obtain a target score map.

[0161] Optionally, the storage medium is further configured to store program code for executing the following steps: when the target scale feature includes multiple sub-features, upsampling the multiple sub-features using a bilinear interpolation method to generate multiple third score maps corresponding to the multiple sub-features; and merging the multiple third score maps to generate a second score map.

[0162] Optionally, the storage medium is further configured to store program code for executing the following steps: obtaining a sample feature vector and a sample likelihood value corresponding to the sample feature vector; estimating the sample feature vector using a target flow model to generate a target likelihood value corresponding to the sample feature vector; obtaining a first distribution corresponding to the sample likelihood value and a second distribution corresponding to the target likelihood value; constructing a target loss function based on the first distribution and the second distribution; and updating the target flow model using the target loss function to obtain an updated result.

[0163] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: displaying a target image on an interactive interface; when a target touch operation is sensed in the interactive interface, displaying a detection result corresponding to the target image in the interactive interface, wherein the detection result is obtained by detecting the target image through a target attention weight and a target scale feature, and the detection result is used to indicate whether a target object exists in the target image, and the target attention weight is determined based on a first scale feature among multiple scale features, and the target scale feature is a scale feature among the multiple scale features other than the first scale feature; the multiple scale features are obtained by performing feature extraction on the target image, and each scale feature corresponds to a different scale.

[0164] An embodiment of the present invention provides an image detection method. First, a target image can be acquired, and features of the target image can be extracted to obtain multiple scale features, wherein each scale feature corresponds to a different scale. Then, based on a first scale feature among the multiple scale features, a target attention weight of the target scale feature is determined, wherein the target scale feature is a scale feature other than the first scale feature among the multiple scale features. Finally, the target image is detected based on the target attention weight and the target scale feature to obtain a detection result. The detection result is used to indicate whether a target object exists in the target image, thereby improving the accuracy of detecting the target object in the target image. Through the attention mechanism, the first scale feature can be used to query feature information of the target scale feature to obtain the target attention weight corresponding to the target scale feature. The target scale features are fused using the target attention weight, and the feature information contained in the multiple scale features can be retained, thereby improving the accuracy of target image detection, thereby solving the technical problem of low accuracy of image anomaly detection in the related art.

[0165] Example 7

[0166] According to an embodiment of the present invention, an image rendering method is also provided. Figure 9 is a flowchart of an image rendering method according to the present application, the method comprising:

[0167] Step S902: Displaying a target image on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device;

[0168] Step S904: extracting features from the target image to obtain multiple scale features, where each scale feature corresponds to a different scale;

[0169] Step S906, determining a target attention weight of a target scale feature based on a first scale feature among the multiple scale features;

[0170] The target scale feature is a scale feature other than the first scale feature among the multiple scale features.

[0171] Step S908: Detect the target image based on the target attention weight and the target scale feature to obtain a detection result;

[0172] The detection result is used to indicate whether the target object exists in the target image.

[0173] Step S910: driving the VR device or AR device to display the detection result.

[0174] The above image rendering method can be applied to Figure 10 In the hardware environment shown, which is composed of the server 102 and the AR-VR device 104, Figure 10 Schematic diagram of the hardware environment of an image rendering method according to an embodiment of the present application. Figure 1 As shown, the server 102 is connected to the AR-VR device 104 via a network, which may be a server corresponding to a media file operator. The network includes but is not limited to a wide area network, a metropolitan area network, or a local area network. The AR-VR device 104 may be a virtual reality VR device or an augmented reality AR device, wherein the virtual reality VR device is not limited to a virtual reality helmet, virtual reality glasses, an all-in-one virtual reality machine, etc.

[0175] Optionally, the AR-VR device 104 includes: a memory, a processor, and a transmission device. The memory is used to store an application, which can be used to perform the following operations: extracting features from a target image to obtain multiple scale features, where each scale feature corresponds to a different scale; determining a target attention weight for a target scale feature based on a first scale feature among the multiple scale features, where the target scale feature is a scale feature other than the first scale feature among the multiple scale features; and detecting the target image based on the target attention weight and the target scale feature to obtain a detection result, where the detection result is used to indicate whether a target object exists in the target image.

[0176] The processor of this embodiment can call the application stored in the memory to execute the above steps through the transmission device. The transmission device can receive the target image sent by the server through the network and can also be used for data transmission between the processor and the memory.

[0177] Optionally, in the AR-VR device 104, a head-mounted display (Head Mount Display, HMD for short) with eye tracking is provided. The screen in the HMD is used to display real-time images. The eye tracking module in the HMD is used to obtain the real-time movement trajectory of the user's eyeballs. The tracking system is used to track the user's position information and motion information in the real three-dimensional space. The computing processing unit is used to obtain the user's real-time position and motion information from the tracking system, and calculate the three-dimensional coordinates of the user's head in the virtual three-dimensional space, as well as the user's field of view direction in the virtual three-dimensional space, etc.

[0178] Figure 11 FIG. 1 is a schematic diagram of a hardware environment for another method for delivering media files according to an embodiment of the present invention. Figure 11As shown, the AR-VR device 104 is connected to the terminal 106, and the terminal 106 is connected to the server 102 through a network. The above-mentioned AR-VR device 104 is not limited to: virtual reality helmets, virtual reality glasses, virtual reality all-in-one machines, etc., and the above-mentioned terminal 104 is not limited to PCs, mobile phones, tablets, etc. The server 102 can be a server corresponding to the media file operator, and the above-mentioned network includes but is not limited to: wide area network, metropolitan area network or local area network.

[0179] Optionally, the AR-VR device 104 of this embodiment has the same function as that in the above embodiment. The terminal of this embodiment can be used to perform: feature extraction on the target image to obtain multiple scale features, wherein each scale feature corresponds to a different scale; based on the first scale feature among the multiple scale features, determining the target attention weight of the target scale feature, wherein the target scale feature is the scale feature other than the first scale feature among the multiple scale features; detecting the target image based on the target attention weight and the target scale feature to obtain a detection result, wherein the detection result is used to indicate whether there is a target object in the target image, and sending the detection result to the AR-VR device 104, which displays the detection result after receiving the detection result.

[0180] Optionally, the AR-VR device 104 of this embodiment includes an eye-tracking HMD and an eye-tracking module that function similarly to those in the aforementioned embodiment. Specifically, the screen in the HMD is used to display real-time images, and the eye-tracking module in the HMD is used to obtain the real-time trajectory of the user's eye movements. The terminal of this embodiment obtains the user's position and movement information in real three-dimensional space through a tracking system, and calculates the three-dimensional coordinates of the user's head in virtual three-dimensional space, as well as the user's field of view in the virtual three-dimensional space.

[0181] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0182] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0183] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0184] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0185] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0186] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), a mobile hard drive, a magnetic disk, or an optical disk.

[0187] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. An image detection method, characterized in that: include: Acquire the target image; Performing feature extraction on the target image to obtain a plurality of scale features, wherein each scale feature corresponds to a different scale; determining a target attention weight of a target scale feature based on a first scale feature among the multiple scale features, wherein the target scale feature is a scale feature other than the first scale feature among the multiple scale features, the first scale feature is a scale feature in an intermediate layer among the multiple scale features, and the target scale feature includes a high-level scale feature and a low-level scale feature; Based on the target attention weight and the target scale feature, generating a likelihood score map corresponding to the high-level scale feature and a likelihood score map corresponding to the low-level scale feature, wherein the likelihood score map is used to represent the resolution of different scale features; A multi-scale aggregation module is used to aggregate the likelihood score map corresponding to the high-level scale features and the likelihood score map corresponding to the low-level scale features to obtain a target score map; The target image is detected based on the target score map to obtain a detection result, wherein the detection result is used to indicate whether a target object exists in the target image.

2. The method according to claim 1, characterized in that Determining a target attention weight of a target scale feature based on a first scale feature among the multiple scale features includes: The target attention weight is obtained based on the product of the first scale feature and the target scale feature.

3. The method according to claim 1, characterized in that Generating a likelihood score map corresponding to the high-level scale feature and a likelihood score map corresponding to the low-level scale feature based on the target attention weight and the target scale feature, including: Generate a target feature vector based on the product of the target attention weight and the target scale feature; Performing likelihood estimation on the target feature vector using a target flow model to generate a target likelihood value corresponding to the target feature vector; Obtain a likelihood score map corresponding to the target likelihood value.

4. The method according to claim 3, characterized in that The scale corresponding to the target score map is the same as that of the target image.

5. The method according to claim 4, characterized in that Detecting the target image based on the target score map to obtain the detection result includes: Processing the target score map based on a preset pixel threshold to obtain a target mask map corresponding to the target score map; The target mask image is detected to obtain the detection result.

6. The method according to claim 3, characterized in that Obtaining a likelihood score map corresponding to the target likelihood value, including: Determining a log-likelihood score corresponding to the target likelihood value based on a log-likelihood formula; A likelihood score map corresponding to the log-likelihood score is determined.

7. The method according to claim 4, characterized in that Processing the likelihood score map to generate a target score map includes: Normalizing the likelihood score map to generate a first score map corresponding to the target scale feature; Upsampling the first score map corresponding to the target scale feature using a bilinear interpolation method to generate a second score map corresponding to the target scale feature, wherein the scale of the second score map is the same as the scale of the scale feature corresponding to the target score map; Normalizing the second score map to obtain the target score map.

8. The method according to claim 7, characterized in that In a case where the target scale feature includes a plurality of sub-features, upsampling the first score map corresponding to the target scale feature by using a bilinear interpolation method to generate a second score map corresponding to the target scale feature includes: Upsampling the plurality of sub-features using the bilinear interpolation method to generate a plurality of third score maps corresponding to the plurality of sub-features; The plurality of third score maps are combined to generate the second score map.

9. The method according to claim 3, characterized in that The method further comprises: Obtaining a sample feature vector and a sample likelihood value corresponding to the sample feature vector; Using the target flow model to estimate the sample feature vector, and generate a target likelihood value corresponding to the sample feature vector; Obtaining a first distribution corresponding to the sample likelihood value and a second distribution corresponding to the target likelihood value; constructing a target loss function based on the first distribution and the second distribution; The target flow model is updated using the target loss function to obtain an updated result.

10. An image detection method, characterized in that: include: Display the target image on the interactive interface; When a target touch operation is sensed in the interactive interface, a detection result corresponding to the target image is displayed in the interactive interface, wherein the detection result is obtained by detecting the target image using a target score map, and the target score map is obtained by aggregating likelihood score maps corresponding to high-level scale features and likelihood score maps corresponding to low-level scale features using a multi-scale aggregation module. The likelihood score maps corresponding to the high-level scale features and the likelihood score maps corresponding to the low-level scale features are obtained based on the target attention weight and the target scale feature, and the target attention weight is determined based on a first scale feature among multiple scale features, where the target scale feature is a scale feature other than the first scale feature among the multiple scale features. The multiple scale features are obtained by performing feature extraction on the target image, each scale feature corresponds to a different scale, the first scale feature is a scale feature in an intermediate layer among the multiple scale features, and the target scale features include high-level scale features and low-level scale features.

11. An image rendering method, characterized in that: include: Displaying a target image on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device; Performing feature extraction on the target image to obtain a plurality of scale features, wherein each scale feature corresponds to a different scale; determining a target attention weight of a target scale feature based on a first scale feature among the multiple scale features, wherein the target scale feature is a scale feature other than the first scale feature among the multiple scale features, the first scale feature is a scale feature in an intermediate layer among the multiple scale features, and the target scale feature includes a high-level scale feature and a low-level scale feature; Based on the target attention weight and the target scale feature, generating a likelihood score map corresponding to the high-level scale feature and a likelihood score map corresponding to the low-level scale feature, wherein the likelihood score map is used to represent the resolution of different scale features; A multi-scale aggregation module is used to aggregate the likelihood score map corresponding to the high-level scale features and the likelihood score map corresponding to the low-level scale features to obtain a target score map; Detecting the target image based on the target score map to obtain a detection result, wherein the detection result is used to indicate whether a target object exists in the target image; The VR device or the AR device is driven to display the detection result.

12. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the device where the storage medium is located is controlled to execute the image detection method according to any one of claims 1 to 10, or the image rendering method according to claim 11.

13. A processor, characterized in that: The processor is used to run a program, wherein the program, when running, executes the image detection method according to any one of claims 1 to 10, or the image rendering method according to claim 11.

Images