Image review method, device and system

The collaborative image auditing method using terminal-side, edge-side, and cloud-side processing addresses server congestion and latency issues by leveraging lightweight models and super-resolution techniques to enhance image filtering accuracy and reduce server load, thereby improving 5G messaging efficiency and user experience.

CN114897888BActive Publication Date: 2025-07-15CHINA TELECOM CORP LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202210710544.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-22
Publication Date
2025-07-15
Estimated Expiration
2042-06-22

AI Technical Summary

Technical Problem

The processing of massive image messages consumes huge computing power on cloud servers, resulting in excessive delay and affecting the user experience of 5G messages.

Method used

The image is initially reviewed using a lightweight model on the terminal side, generating thumbnails and calculating confidence; transmitting images with low confidence to the edge for super-resolution processing; and finally using models with powerful computing power to perform accurate review in the cloud.

Benefits of technology

It reduces the processing pressure of cloud servers, reduces latency, and improves the user experience of 5G message users.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114897888B_ABST
    Figure CN114897888B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a method, apparatus and system for image review, and relates to the field of communication technologies. The review method includes: judging whether the image to be reviewed is a spam image according to the relevant information of the review image on the terminal side; in the case where the judgment result is that the image is not a spam image, sending the relevant information of the image to be reviewed to the cloud so that the cloud can judge whether the image to be reviewed is a spam image. The technical solution of the present disclosure can reduce the processing pressure of the server, thereby reducing the latency and improving the user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of communication technologies, and particularly to an image review method, an image review device, an image review system, and a non-volatile computer-readable storage medium. Background Art

[0002] Junk message filtering is a key link for the stable and secure operation of the 5G messaging platform, which helps to improve the service quality of mobile users. The more accurate the junk message filtering is, the more stable the platform is and the better the user experience is.

[0003] In the related art, data is transmitted to a server in the cloud, and after being identified, it is determined whether filtering is required. Summary of the Invention

[0004] The inventors of the present disclosure have found the following problems in the above-mentioned related art: Processing a large number of picture messages requires consuming huge computing resources, which is extremely likely to cause congestion in the cloud server, resulting in excessive latency during the message transmission process and reducing the user experience of 5G messaging.

[0005] In view of this, the present disclosure proposes an image review technical solution, which can reduce the processing pressure on the server, thereby reducing latency and improving the user experience.

[0006] According to some embodiments of the present disclosure, there is provided an image review method, including: judging whether a to-be-reviewed image is a junk image according to relevant information of the review image on the terminal side; in the case that the judgment result is a non-junk image, sending the relevant information of the to-be-reviewed image to the cloud so that the cloud judges whether the to-be-reviewed image is a junk image.

[0007] In some embodiments, the relevant information of the to-be-reviewed image includes a thumbnail of the to-be-reviewed image, and judging whether the to-be-reviewed image is a junk image includes: generating a thumbnail of the to-be-reviewed image on the terminal side; judging whether the to-be-reviewed image is a junk image according to the thumbnail of the review image.

[0008] In some embodiments, sending the relevant information of the to-be-reviewed image to the cloud includes: sending the thumbnail to the edge side so that the edge side performs super-resolution processing on the thumbnail and then forwards it to the cloud for review.

[0009] In some embodiments, judging whether the to-be-reviewed image is a junk image includes: using a lightweight model deployed on the terminal side to calculate a first confidence level that the to-be-reviewed image belongs to a junk image; in the case that the first confidence level is greater than a first threshold, judging that the to-be-reviewed image is a junk image; in the case that the first confidence level is less than or equal to the first threshold, judging that the to-be-reviewed image is a non-junk image.

[0010] In some embodiments, the cloud uses the YOLOv5 model to determine whether the image to be reviewed is a junk image. Determining whether the image to be reviewed is a junk image includes: using the MobileNet v3 (Mobile Network Version 3) model on the terminal side to determine whether the image to be reviewed is a junk image.

[0011] According to some other embodiments of the present disclosure, there is provided a method for reviewing an image, including: when it is determined on the calling terminal that the image to be reviewed is not a junk image, on the cloud side, determining whether the image to be reviewed is a junk image according to the relevant information of the image to be reviewed sent by the calling terminal; when the determination result is that the image is not a junk image, sending the relevant information of the image to be reviewed to the called terminal.

[0012] In some embodiments, the relevant information of the image to be reviewed includes a thumbnail of the image to be reviewed. Determining whether the image to be reviewed is a junk image according to the relevant information of the image to be reviewed sent by the calling terminal includes: receiving the super-resolution image obtained after the edge side performs super-resolution processing on the thumbnail; determining whether the image to be reviewed is a junk image according to the super-resolution image.

[0013] In some embodiments, determining whether the image to be reviewed is a junk image according to the super-resolution image includes: using the image recognition model deployed on the cloud side to calculate the second confidence level that the image to be reviewed belongs to a junk image; when the second confidence level is greater than the second threshold, determining that the image to be reviewed is a junk image; when the second confidence level is less than or equal to the second threshold, determining that the image to be reviewed is not a junk image.

[0014] In some embodiments, the relevant information of the image to be reviewed includes the super-resolution image of the image to be reviewed. Sending the relevant information of the image to be reviewed to the called terminal includes: sending the super-resolution image to the called terminal.

[0015] In some embodiments, the cloud uses the MobileNet v3 model to determine whether the image to be reviewed is a junk image. Determining whether the image to be reviewed is a junk image includes: using the YOLOv5 model on the cloud side to determine whether the image to be reviewed is a junk image.

[0016] According to some further embodiments of the present disclosure, there is provided an image review device, including: a determination unit for determining whether the image to be reviewed is a junk image according to the relevant information of the reviewed image on the terminal side; a sending unit for, when the determination result is that the image is not a junk image, sending the relevant information of the image to be reviewed to the cloud so that the cloud can determine whether the image to be reviewed is a junk image.

[0017] In some embodiments, the determination unit generates a thumbnail of the image to be reviewed on the terminal side, and determines whether the image to be reviewed is a junk image according to the thumbnail of the reviewed image.

[0018] In some embodiments, the sending unit sends the thumbnail to the edge side so that the edge side performs super-resolution processing on the thumbnail and then forwards it to the cloud for review.

[0019] In some embodiments, the judging unit uses a lightweight model deployed on the terminal side to calculate a first confidence level that the image to be reviewed belongs to a junk image. When the first confidence level is greater than a first threshold, it is judged that the image to be reviewed is a junk image. When the first confidence level is less than or equal to the first threshold, it is judged that the image to be reviewed is a non-junk image.

[0020] In some embodiments, the judging unit uses the MobileNet v3 model on the terminal side to judge whether the image to be reviewed is a junk image.

[0021] According to still some other embodiments of the present disclosure, there is provided an image review device, including: a judging unit, configured to, when it is judged on the calling terminal side that the image to be reviewed is a non-junk image, judge whether the image to be reviewed is a junk image according to the relevant information of the image to be reviewed sent by the calling terminal on the cloud side; a sending unit, configured to, when the judgment result is a non-junk image, send the relevant information of the image to be reviewed to the called terminal.

[0022] In some embodiments, the review device further includes a receiving unit, configured to receive the super-resolution image obtained by the edge side after performing super-resolution processing on the thumbnail; and the judging unit judges whether the image to be reviewed is a junk image according to the super-resolution image.

[0023] In some embodiments, the judging unit uses an image recognition model deployed on the cloud side to calculate a second confidence level that the image to be reviewed belongs to a junk image. When the second confidence level is greater than a second threshold, it is judged that the image to be reviewed is a junk image. When the second confidence level is less than or equal to the second threshold, it is judged that the image to be reviewed is a non-junk image.

[0024] In some embodiments, the relevant information of the image to be reviewed includes the super-resolution image of the image to be reviewed, and the sending unit sends the super-resolution image to the called terminal.

[0025] In some embodiments, the judging unit uses the yolov5 model on the cloud side to judge whether the image to be reviewed is a junk image.

[0026] According to still some other embodiments of the present disclosure, there is provided an image review system, including: a terminal-side device, configured to execute the image review method on the terminal side in any one of the above embodiments; a cloud-side device, configured to execute the image review method on the cloud side in any one of the above embodiments.

[0027] In some embodiments, the audit system further includes: an edge device, configured to perform super-resolution processing on a thumbnail of a to-be-audited image sent by a terminal-side device, and forward the super-resolution processing result to a cloud-side device.

[0028] According to still other embodiments of the present disclosure, there is provided an image audit apparatus, including: a memory; and a processor coupled to the memory, the processor being configured to execute the image audit method in any of the above embodiments based on instructions stored in the memory device.

[0029] According to still other embodiments of the present disclosure, there is provided a non-volatile computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the image audit method in any of the above embodiments is implemented.

[0030] In the above embodiments, the to-be-audited image is pre-audited on the terminal side, and only the to-be-audited images that pass the audit are sent to the cloud side for re-audit. In this way, the processing pressure on the server can be reduced, thereby reducing the latency and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings forming a part of the specification depict embodiments of the present disclosure and, together with the specification, are used to explain the principles of the present disclosure.

[0032] Referring to the drawings, the present disclosure can be more clearly understood from the following detailed description, where:

[0033] Figure 1 Flowcharts showing some embodiments of the image audit method of the present disclosure;

[0034] Figures 2a - 2b Flowcharts showing other embodiments of the image audit method of the present disclosure;

[0035] Figure 3 Flowcharts showing still other embodiments of the image audit method of the present disclosure;

[0036] Figure 4 Block diagrams showing some embodiments of the image audit apparatus of the present disclosure;

[0037] Figure 5 Block diagrams showing other embodiments of the image audit apparatus of the present disclosure;

[0038] Figure 6 Block diagrams showing still other embodiments of the image audit apparatus of the present disclosure;

[0039] Figure 7 Block diagrams showing some embodiments of the image audit system of the present disclosure. DETAILED DESCRIPTION

[0040] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that: Unless otherwise specifically stated, the relative arrangements, numerical expressions, and numerical values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0041] Meanwhile, it should be understood that, for the sake of convenience in description, the dimensions of the respective parts shown in the drawings are not drawn in actual proportional relationship.

[0042] The following description of at least one exemplary embodiment is merely illustrative in nature and in no way serves as a limitation to the present disclosure and its application or use.

[0043] Techniques, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the said techniques, methods, and devices should be regarded as part of the specification.

[0044] In all the examples shown and discussed here, any specific value should be construed as merely exemplary and not as a limitation. Thus, other examples of the exemplary embodiments may have different values.

[0045] It should be noted that: Similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0046] As mentioned above, for massive data pictures, adopting the method of thumbnail transmission will improve the transmission efficiency. However, it will also reduce the recognition accuracy of the cloud server, resulting in a large number of misjudgments of spam messages, interfering with the normal use of 5G message users.

[0047] To address the above technical problems, the present disclosure proposes a collaborative review method for thumbnails based on super-resolution. To address the problems of low platform picture recognition accuracy and long review time, a collaborative review strategy based on super-resolution for cloud-edge-terminal is proposed.

[0048] In some embodiments, a lightweight model is deployed on the mobile phone side, and the basic computing power of the terminal is utilized to first review the thumbnail and calculate the confidence level; the pictures with low confidence levels are transmitted to the edge side, and the thumbnail is optimized using super-resolution to obtain a higher-definition picture; finally, the picture is pushed from the edge side to the cloud, and the powerful computing power of the cloud is utilized to perform recognition using the yolov5 network model to obtain an accurate picture recognition result, completing the picture collaborative review process. For example, the technical solution of the present disclosure can be implemented through the following embodiments.

[0049] Figure 1 The flowchart showing some embodiments of the image review method of the present disclosure.

[0050] As shown Figure 1 In step 110, on the terminal side, based on the relevant information of the image to be audited, it is determined whether the image to be audited is a junk image.

[0051] In some embodiments, a thumbnail of the image to be audited is generated on the terminal side; based on the thumbnail of the image to be audited, it is determined whether the image to be audited is a junk image.

[0052] For example, during the process of sending a picture by the user terminal, the picture is converted into a thumbnail to improve the transmission rate.

[0053] In some embodiments, a lightweight model deployed on the terminal side is used to calculate the first confidence level that the image to be audited belongs to a junk image; in the case where the first confidence level is greater than the first threshold, it is determined that the image to be audited is a junk image; in the case where the first confidence level is less than or equal to the first threshold, it is determined that the image to be audited is a non-junk image.

[0054] In some embodiments, determining whether the image to be audited is a junk image includes: using the MobileNetv3 model on the terminal side to determine whether the image to be audited is a junk image.

[0055] For example, a lightweight model mobilenet v3 is deployed on the terminal side; the model is used to identify the thumbnail and calculate the confidence level of whether the picture is compliant; a confidence level greater than 80% indicates that the model believes the picture is non-compliant and needs to be filtered; a confidence level less than 80% indicates that the picture is compliant; considering the small computing power of the terminal and the low clarity of the picture, the pictures with a confidence level less than 80% are sent to the cloud for further identification.

[0056] In step 120, in the case where the judgment result is a non-junk image, the relevant information of the image to be audited is sent to the cloud so that the cloud can determine whether the image to be audited is a junk image.

[0057] In some embodiments, the thumbnail is sent to the edge side so that the edge side can perform super-resolution processing on the thumbnail and then forward it to the cloud for auditing.

[0058] For example, a lightweight super-resolution model is adopted on the edge side to improve the image clarity and send the improved picture to the cloud.

[0059] In some embodiments, the cloud uses the yolov5 model to determine whether the image to be audited is a junk image.

[0060] For example, the cloud uses the yolov5 model to identify a clear picture (such as a super-resolution image) and calculate the confidence level of the picture being compliant; if the confidence level is greater than 60%, it is classified as a junk picture; if the confidence level is less than 60%, it is classified as a compliant picture; the compliant pictures with a confidence level less than 60% in the model recognition result are forwarded to the called terminal.

[0061] In the above embodiments, based on super-resolution and cloud-edge-terminal collaborative review of the thumbnail sent by the end user, the review pressure on the cloud server is reduced, the stability of the server is improved, and the usage experience of 5G message users is enhanced.

[0062] Figures 2a - 2b The flowchart showing some other embodiments of the image review method of the present disclosure.

[0063] As Figure 2a shown, based on the hardware resources and computing resources of the edge side and the cloud, this system architecture is deployed on the edge side and the cloud. The system includes a terminal lightweight image recognition module, an edge-side super-resolution module, a cloud image recognition module, etc.

[0064] The lightweight image recognition module of the calling terminal A on the terminal side utilizes the computing power of the terminal to be responsible for the preliminary review of the thumbnail sent by the user.

[0065] The super-resolution module on the edge side is responsible for processing the thumbnails that cannot be distinguished by the terminal and improving the clarity of the thumbnails.

[0066] The image recognition module in the cloud is responsible for the final review of the images transmitted by the user. This module uses the YOLOv5 model and has a high recognition accuracy. The cloud sends the super-resolution image of the image that has passed its review to the called terminal B.

[0067] As Figure 2b shown, during the process of the user terminal (terminal A) sending a picture, the picture is converted into a thumbnail to improve the transmission rate.

[0068] A lightweight model mobilenet v3 is deployed on the terminal side. This model is used to recognize the thumbnail and calculate the confidence level of whether the picture is compliant. A confidence level greater than 80% indicates that the model believes the picture is non-compliant and needs to be filtered; a confidence level less than 80% indicates that the picture is compliant; considering the small computing power of the terminal and the low clarity of the picture, the pictures with a confidence level less than 80% are sent to the cloud for further recognition.

[0069] The RAISR super-resolution model uses a lightweight super-resolution model on the edge side to improve the image clarity and sends the improved picture to the cloud.

[0070] The cloud uses the yolov5 model to recognize the clear picture and calculate the confidence level of the picture being compliant; a confidence level greater than 60% is classified as a junk picture; a confidence level less than 60% is classified as a compliant picture.

[0071] Forward the compliant pictures with a confidence level less than 60% in the model recognition result to the called terminal B.

[0072] Figure 3Flowchart showing further embodiments of the method for auditing images of the present disclosure.

[0073] As Figure 3 shown, in step 310, when the calling terminal determines that the image to be audited is not a junk image, the cloud measures whether the image to be audited is a junk image according to the relevant information of the image to be audited sent by the calling terminal.

[0074] In some embodiments, the relevant information of the image to be audited includes the thumbnail of the image to be audited. Receive the super-resolution image obtained after the edge side performs super-resolution processing on the thumbnail; determine whether the image to be audited is a junk image according to the super-resolution image.

[0075] In some embodiments, use the image recognition model deployed by the cloud to calculate the second confidence level that the image to be audited belongs to a junk image; when the second confidence level is greater than the second threshold, determine that the image to be audited is a junk image; when the second confidence level is less than or equal to the second threshold, determine that the image to be audited is not a junk image.

[0076] In some embodiments, the cloud uses the MobileNet v3 model to determine whether the image to be audited is a junk image. On the cloud side, use the yolov5 model to determine whether the image to be audited is a junk image.

[0077] In step 320, when the judgment result is that the image is not a junk image, send the relevant information of the image to be audited to the called terminal.

[0078] In some embodiments, the relevant information of the image to be audited includes the super-resolution image of the image to be audited. Send the super-resolution image to the called terminal.

[0079] In the above embodiments, the collaborative auditing method based on the super-resolution thumbnail adopts the method of cloud-edge-terminal collaborative auditing, reduces the thumbnail auditing time in the 5G message platform, and improves the thumbnail auditing accuracy. The image recognition module (terminal side) deploys the lightweight image recognition model mobilenetv3 on the terminal, and the terminal user will automatically audit the thumbnail of the image when sending the image.

[0080] In this way, deploy a lightweight super-resolution model at the edge side and a lightweight model mobilenet V3 at the terminal side, without involving hardware transformation, and has good practicability.

[0081] Figure 4 Block diagram showing some embodiments of the image auditing device of the present disclosure.

[0082] As Figure 4As shown, in some embodiments, the image review device 4 includes: a judgment unit 41, configured to judge whether the image to be reviewed is a junk image according to the relevant information of the reviewed image on the terminal side; a sending unit 42, configured to, when the judgment result is that the image is not a junk image, send the relevant information of the image to be reviewed to the cloud, so that the cloud can judge whether the image to be reviewed is a junk image.

[0083] In some embodiments, the judgment unit 41 generates a thumbnail of the image to be reviewed on the terminal side, and judges whether the image to be reviewed is a junk image according to the thumbnail of the reviewed image.

[0084] In some embodiments, the sending unit 42 sends the thumbnail to the edge side, so that the edge side performs super-resolution processing on the thumbnail and then forwards it to the cloud for review.

[0085] In some embodiments, the judgment unit 41 uses a lightweight model deployed on the terminal side to calculate a first confidence level that the image to be reviewed belongs to a junk image. When the first confidence level is greater than a first threshold, it is judged that the image to be reviewed is a junk image. When the first confidence level is less than or equal to the first threshold, it is judged that the image to be reviewed is not a junk image.

[0086] In some embodiments, the judgment unit 41 uses the MobileNet v3 model on the terminal side to judge whether the image to be reviewed is a junk image.

[0087] In some embodiments, the image review device 4 includes: a judgment unit 41, configured to, when it is judged on the calling terminal that the image to be reviewed is not a junk image, judge whether the image to be reviewed is a junk image according to the relevant information of the image to be reviewed sent by the calling terminal on the cloud side; a sending unit 42, configured to, when the judgment result is that the image is not a junk image, send the relevant information of the image to be reviewed to the called terminal.

[0088] In some embodiments, the review device 4 further includes a receiving unit 43, configured to receive the super-resolution image obtained by the edge side after performing super-resolution processing on the thumbnail; the judgment unit 41 judges whether the image to be reviewed is a junk image according to the super-resolution image.

[0089] In some embodiments, the judgment unit 41 uses an image recognition model deployed on the cloud side to calculate a second confidence level that the image to be reviewed belongs to a junk image. When the second confidence level is greater than a second threshold, it is judged that the image to be reviewed is a junk image. When the second confidence level is less than or equal to the second threshold, it is judged that the image to be reviewed is not a junk image.

[0090] In some embodiments, the relevant information of the image to be reviewed includes the super-resolution image of the image to be reviewed, and the sending unit 42 sends the super-resolution image to the called terminal.

[0091] In some embodiments, the determination unit 41 uses the yolov5 model on the cloud side to determine whether the image to be reviewed is a junk image.

[0092] Figure 5 A block diagram showing other embodiments of the image review device of the present disclosure.

[0093] As Figure 5 As shown, the image review device 5 of this embodiment includes: a memory 51 and a processor 52 coupled to the memory 51. The processor 52 is configured to execute the image review method in any one of the embodiments of the present disclosure based on the instructions stored in the memory 51.

[0094] Among them, the memory 51 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory stores, for example, an operating system, application programs, a boot loader, a database, and other programs.

[0095] Figure 6 A block diagram showing still other embodiments of the image review device of the present disclosure.

[0096] As Figure 6 As shown, the image review device 6 of this embodiment includes: a memory 610 and a processor 620 coupled to the memory 610. The processor 620 is configured to execute the image review method in any of the foregoing embodiments based on the instructions stored in the memory 610.

[0097] The memory 610 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory stores, for example, an operating system, application programs, a boot loader, and other programs.

[0098] The image review device 6 may further include an input / output interface 630, a network interface 640, a storage interface 650, etc. These interfaces 630, 640, 650 and the memory 610 and the processor 620 may be connected through a bus 660, for example. Among them, the input / output interface 630 provides a connection interface for input / output devices such as a display, a mouse, a keyboard, a touch screen, a microphone, and a speaker. The network interface 640 provides a connection interface for various networking devices. The storage interface 650 provides a connection interface for external storage devices such as an SD card and a USB flash drive.

[0099] Figure 7 A block diagram showing some embodiments of the image review system of the present disclosure.

[0100] As Figure 7As shown in the figure, the image review system 7 includes: a terminal-side device 71 for executing the terminal-side image review method in any of the above embodiments; and a cloud-side device 72 for executing the cloud-side image review method in any of the above embodiments.

[0101] In some embodiments, the review system 7 further includes: an edge-side device 73 for performing super-resolution processing on the thumbnail of the image to be reviewed sent by the terminal-side device and forwarding the super-resolution processing result to the cloud-side device.

[0102] Those skilled in the art should understand that the embodiments of the present disclosure can be provided as a method, a system, or a computer program product. Therefore, the present disclosure can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure can take the form of a computer program product implemented on one or more computer-usable non-transitory storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] So far, the image review method, the image review device, the image review system, and the non-volatile computer-readable storage medium according to the present disclosure have been described in detail. To avoid obscuring the concept of the present disclosure, some details well known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed here based on the above description.

[0104] The methods and systems of the present disclosure can be implemented in many ways. For example, the methods and systems of the present disclosure can be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is only for illustration, and the steps of the method of the present disclosure are not limited to the specific order described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure can also be implemented as a program recorded in a recording medium, and these programs include machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0105] Although some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art should understand that the above examples are only for illustration and not for limiting the scope of the present disclosure. Those skilled in the art should understand that the above embodiments can be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. An image review method, comprising: On the terminal side, based on the relevant information of the image to be reviewed, determining whether the image to be reviewed is a junk image; In the case where the determination result is that the image is not a junk image, sending the relevant information of the image to be reviewed to the cloud, so that the cloud determines whether the image to be reviewed is a junk image, wherein, the relevant information of the image to be reviewed includes the thumbnail of the image to be reviewed, The determination of whether the image to be reviewed is a junk image includes: Generating a thumbnail of the image to be reviewed on the terminal side; Based on the thumbnail of the reviewed image, determining whether the image to be reviewed is a junk image, wherein, the sending the relevant information of the image to be reviewed to the cloud includes: Sending the thumbnail to the edge side, so that the edge side performs super-resolution processing on the thumbnail and forwards it to the cloud for review.

2. The auditing method according to claim 1, wherein, The determination of whether the image to be reviewed is a junk image includes: Using the lightweight model deployed on the terminal side to calculate the first confidence level that the image to be reviewed belongs to a junk image; In the case where the first confidence level is greater than the first threshold, determining that the image to be reviewed is a junk image; In the case where the first confidence level is less than or equal to the first threshold, determining that the image to be reviewed is not a junk image.

3. The review method according to claim 1 or 2, wherein The cloud uses the yolov5 model to determine whether the image to be reviewed is a junk image, The determination of whether the image to be reviewed is a junk image includes: On the terminal side, using the MobileNet v3 model of the third-generation mobile network to determine whether the image to be reviewed is a junk image.

4. An image review method, comprising: In the case where the calling terminal determines that the image to be reviewed is not a junk image, on the cloud side, based on the relevant information of the image to be reviewed sent by the calling terminal, determining whether the image to be reviewed is a junk image; In the case where the determination result is that the image is not a junk image, sending the relevant information of the image to be reviewed to the called terminal, wherein, the relevant information of the image to be reviewed includes the thumbnail of the image to be reviewed, The determination of whether the image to be reviewed is a junk image based on the relevant information of the image to be reviewed sent by the calling terminal includes: Receiving the super-resolution image obtained by the edge side after performing super-resolution processing on the thumbnail; Based on the super-resolution image, determining whether the image to be reviewed is a junk image.

5. The review method according to claim 4, wherein, The determination of whether the image to be reviewed is a junk image based on the super-resolution image includes: Using the image recognition model deployed on the cloud side to calculate the second confidence level that the image to be reviewed belongs to a junk image; In the case where the second confidence level is greater than the second threshold, determining that the image to be reviewed is a junk image; In the case where the second confidence level is less than or equal to the second threshold, determining that the image to be reviewed is not a junk image.

6. The auditing method according to claim 4, wherein, The relevant information of the image to be reviewed includes the super-resolution image of the image to be reviewed, The sending the relevant information of the image to be reviewed to the called terminal includes: Sending the super-resolution image to the called terminal.

7. The auditing method according to any one of claims 4-6, wherein, The cloud uses the MobileNet v3 model of the third-generation mobile network to determine whether the image to be reviewed is a junk image. The determination of whether the image to be reviewed is a junk image includes: On the cloud side, use the yolov5 model to determine whether the image to be reviewed is a junk image.

8. An image review device, comprising: A judgment unit, configured to judge whether the image to be reviewed is a junk image according to the relevant information of the image to be reviewed on the terminal side; A sending unit, configured to send the relevant information of the image to be reviewed to the cloud when the judgment result is that the image is not a junk image, so that the cloud determines whether the image to be reviewed is a junk image. Wherein, the relevant information of the image to be reviewed includes the thumbnail of the image to be reviewed. The judgment unit generates the thumbnail of the image to be reviewed on the terminal side, and judges whether the image to be reviewed is a junk image according to the thumbnail of the reviewed image. The sending unit sends the thumbnail to the edge side, so that the edge side performs super-resolution processing on the thumbnail and forwards it to the cloud for review.

9. An image review device, comprising: A judgment unit, configured to judge whether the image to be reviewed is a junk image according to the relevant information of the image to be reviewed sent by the calling terminal on the cloud side when it is judged that the image to be reviewed on the calling terminal is not a junk image; A sending unit, configured to send the relevant information of the image to be reviewed to the called terminal when the judgment result is that the image is not a junk image, and the relevant information of the image to be reviewed includes the thumbnail of the image to be reviewed; A receiving unit, configured to receive the super-resolution image obtained by performing super-resolution processing on the thumbnail by the edge side; Wherein, the judgment unit judges whether the image to be reviewed is a junk image according to the super-resolution image.

10. An image review system, comprising: Terminal-side device, configured to execute the image review method according to any one of claims 1 to 3; Cloud-side device, configured to execute the image review method according to any one of claims 4 to 7.

11. The review system according to claim 10, further comprising: Edge-side device, configured to perform super-resolution processing on the thumbnail of the image to be reviewed sent by the terminal-side device and forward the super-resolution processing result to the cloud-side device.

12. An image review device, comprising: A memory; And A processor coupled to the memory, the processor being configured to execute the image review method according to any one of claims 1 to 3, or the image review method according to any one of claims 4 to 7 based on instructions stored in the memory.

13. A non-volatile computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the image review method according to any one of claims 1 to 3, or the image review method according to any one of claims 4 to 7.

Citation Information

Patent Citations

  • Image recognition method, device and apparatus

    CN109919109A

  • Intelligent garbage classification method and system

    CN110929693A

  • Video data transmission method and device, video data processing method and device, and electronic equipment

    CN111405296A

  • Remote examination control method, server and terminal

    CN111444899A