Picture auditing optimization method and device and computer readable storage medium

Abstract

The invention discloses a picture auditing optimization method and device and a computer readable storage medium. The picture auditing optimization method comprises the following steps: automatically auditing extracted pictures in sequence according to a time sequence; monitoring computing power resources of picture auditing, stopping automatic auditing of the pictures when the computing power resources exceed a preset threshold value, extracting the pictures in a sequence opposite to the time sequence, and then automatically auditing the pictures; according to the method, all identification resources can be ensured to process new video data preferentially, it is ensured that automatic identification can be better than that before manual auditing, automatic auditing after manual auditing is avoided, and waste of invalid resources is prevented.

Description

Technical Field

[0001] The present invention relates to the field of picture review, and in particular to an optimized method, device and computer-readable storage medium for picture review. Background Art

[0002] In current Internet systems, the number of video accounts or various surveillance videos is very large. To meet relevant regulatory requirements, manual review operations of videos are usually required.

[0003] To slow down the manual review operation, there are currently some automated recognition operations. Some content in the video is frame-extracted to generate pictures, and the picture content and text are automatically reviewed. Finally, corresponding tags are set for the video according to certain rules, including high-risk, medium-risk, low-risk and other tags. After the automatic review is completed and the tags are set, the manual operation can be selectively carried out. Therefore, the speed of automatic review needs to be faster than that of manual review. If the manual review of the latest data has been completed, and then the automatic review is carried out again, it will lead to ineffective waste of resources. However, in the current automatic review process, all tasks are carried out in sequence. When tasks pile up, subsequent tasks cannot be recognized as soon as possible. At this time, if the manual operation gives priority to processing the latest data and then the automatic review program conducts recognition again, it will lead to waste of resources. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an optimized method, device and computer-readable storage medium for picture review to avoid waste of resources in the process of image review.

[0005] To solve the above technical problem, a technical solution adopted by the present invention is: An optimized method for picture review, comprising the steps of: Automatically review the extracted pictures in chronological order; Monitor the computing power resources for picture review. When the computing power resources exceed a preset threshold, stop the automatic review of the pictures, extract the pictures in the reverse order of the time sequence and then conduct the automatic review.

[0006] To solve the above technical problem, another technical solution adopted by the present invention is: An optimized device for picture review, comprising a memory, a processor and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned optimized method for picture review are implemented.

[0007] To solve the above technical problem, another technical solution adopted by the present invention is: A computer-readable storage medium stores computer program instructions thereon, and when the computer program instructions are executed by a processor, the steps of the above-mentioned optimization method for picture review are implemented.

[0008] The beneficial effects of the present invention are as follows: When automatically reviewing pictures, the pictures are automatically reviewed in a first-in, first-out manner, and monitoring of the computing power resources for picture review is added. When the computing power resources exceed a preset threshold, the automatic review of the pictures is stopped, and the pictures are extracted and then automatically reviewed in a last-in, first-out order. Through the above method, it can be ensured that all recognition resources give priority to processing newer video data, ensure that automatic recognition can be better than manual review before, avoid automatic review after manual review, and prevent waste of invalid resources. Description of the Drawings

[0009] Figure 1 It is a flowchart of the steps of an optimization method for picture review according to an embodiment of the present invention; Figure 2 It is a schematic structural diagram of an optimization device for picture review according to an embodiment of the present invention. Detailed Embodiments

[0010] To describe in detail the technical content, the achieved objectives and the effects of the present invention, the following is described in conjunction with the embodiments and with reference to the drawings.

[0011] Please refer to Figure 1 , an optimization method for picture review, including the steps: Automatically review the extracted pictures in chronological order; Monitor the computing power resources for picture review. When the computing power resources exceed a preset threshold, stop the automatic review of the pictures, and extract the pictures in the order opposite to the chronological order and then conduct an automatic review.

[0012] As can be seen from the above description, the beneficial effects of the present invention are as follows: When automatically reviewing pictures, the pictures are automatically reviewed in a first-in, first-out manner, and monitoring of the computing power resources for picture review is added. When the computing power resources exceed a preset threshold, the automatic review of the pictures is stopped, and the pictures are extracted and then automatically reviewed in a last-in, first-out order. Through the above method, it can be ensured that all recognition resources give priority to processing newer video data, ensure that automatic recognition can be better than manual review before, avoid automatic review after manual review, and prevent waste of invalid resources.

[0013] Further, the automatically reviewing the extracted pictures in chronological order includes: Perform random frame extraction on the video to be audited, generate random frame extraction pictures, and send the frame extraction pictures to the message queue; Extract pictures from the message queue in sequence for automatic auditing.

[0014] As can be seen from the above description, with the help of the message queue, it is possible to efficiently and quickly extract pictures in chronological order.

[0015] Further, the extraction of pictures in the order opposite to the chronological order includes: Write the pictures extracted in chronological order into the zset structure of redis; Extract pictures from the zset structure.

[0016] As can be seen from the above description, with the help of the zset structure of redis, it is possible to efficiently and quickly extract pictures in reverse chronological order.

[0017] Further, each extracted picture has a corresponding message body content, and the message body content includes a timestamp; When writing the picture into the zset structure of redis, use the score field to record the timestamp corresponding to the picture; When extracting pictures from the zset structure, extract the pictures in reverse chronological order according to the timestamp recorded in the score field.

[0018] As can be seen from the above description, the extracted pictures have message body content including timestamps. By using the score field of the zset structure to record the timestamps corresponding to the pictures, it is convenient and fast to extract the pictures in reverse order according to the timestamps of the score field during picture extraction.

[0019] Further, when the computing power resource resumes below the preset threshold, then resume the step of automatically auditing the extracted pictures in chronological order.

[0020] As can be seen from the above description, when the computing power resource resumes, then resume the traditional operation of automatic auditing plus manual auditing to achieve the orderly progress of picture auditing.

[0021] Further, the step of resuming the automatic auditing of the extracted pictures in chronological order includes: Judge the timeliness of the currently extracted picture; If the timeliness of the picture is lower than the preset timeliness, then abandon the automatic auditing of the picture and start the automatic auditing from the pictures that meet the preset timeliness.

[0022] As described above, when restoring the automatic review operation, first determine the timeliness of the currently extracted picture. If the timeliness has passed, directly discard it to further save computing resources.

[0023] Furthermore, each extracted picture has a corresponding message body content, and the message body content includes a picture serial number; The determination of the timeliness of the currently extracted picture includes: Determine the timeliness of the currently extracted picture according to the picture serial number corresponding to the currently extracted picture.

[0024] As described above, it is convenient to determine the timeliness of a picture by means of the picture serial number included in the message body content of the picture.

[0025] Furthermore, the automatic review of the extracted pictures includes: Automatically recognize the picture content and text of the extracted picture, and determine whether there is sensitive content. If so, determine its risk level according to the sensitive content; Set a corresponding risk level label for the picture based on the determined risk level.

[0026] As described above, during the automatic review process, by automatically recognizing the picture content and text of the picture and attaching the corresponding risk level label, it is convenient for subsequent verification and confirmation by manual review.

[0027] Please refer to Figure 2 , an optimization device for picture review, includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the above-mentioned optimization method for picture review.

[0028] A computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, they implement the steps of the above-mentioned optimization method for picture review.

[0029] The above-mentioned optimization method, device, and computer-readable storage medium for picture review of the present application can be applied to scenarios that require automatic picture review. The following is an illustration through specific embodiments: In an alternative embodiment, please refer to Figure 1 , an optimization method for picture review, includes the steps of: S1. Sequentially perform automatic review on the extracted pictures in chronological order. Random frame extraction actions can be performed on all videos to generate random frame extraction pictures, and a corresponding task message id is generated for each picture; Among them, the automatic review of the extracted pictures includes: Automatically identify the content and text of the extracted pictures, determine whether there is sensitive content, and if so, determine its risk level according to the sensitive content; Set a corresponding risk level label for the picture based on the determined risk level; S2. Monitor the computing resources for picture review. When the computing resources exceed the preset threshold, stop the automatic review of the picture, extract the pictures in the reverse order of time sequence and then conduct automatic review. In a preferred embodiment, the computing resources for picture review can be monitored in real time.

[0030] In another alternative embodiment, the step of sequentially performing automatic review on the extracted pictures in chronological order includes: Perform a random frame extraction action on the video to be reviewed, generate random frame-extracted pictures, and send the frame-extracted pictures to the message queue; Send the task message corresponding to step S1 to MQ, and the content of the message body includes: video id + picture serial number + picture address, etc.; Extract pictures from the message queue in sequence for automatic review; In specific implementation, there are multiple MQ consumers, which will regularly consume messages from MQ. After consuming the messages, they will call the picture recognition method to perform subsequent automatic recognition and review operations. When there are problems or risks, corresponding risk levels will be set for different pictures.

[0031] In another alternative embodiment, the step of extracting pictures in the reverse order of time sequence includes: Write the pictures sequentially extracted in chronological order into the zset structure of Redis; Extract pictures from the zset structure.

[0032] Among them, each extracted picture has a corresponding message body content, and the message body content includes a timestamp; When writing the picture into the zset structure of Redis, use the score field to record the timestamp corresponding to the picture; When extracting pictures from the zset structure, extract the pictures in reverse chronological order according to the timestamp recorded in the score field; The newly added monitoring program will monitor the computing resources for picture recognition. When the threshold is exceeded, the special process switch will be turned on. At this time, after the consumers in MQ consume the messages, they will not call the picture recognition method, but directly write the task messages into the zset structure of Redis, and the score field uses the timestamp field in the message; There is also a timing task that regularly reads messages from the zset structure in redis. It reads messages in reverse order of the score. Since the score is in reverse order of time, at this time, the task messages being processed are all the latest task messages, and the remaining earlier messages will be processed later; When the computing resources are tight within a certain period of time, the latest messages are processed first for machine recognition. At this time, for earlier messages, they can be abandoned or continue to wait for processing. At this time, manual review is still carried out in the order of the time of the messages. In this way, the number of machine recognition times can be reduced because, in this way, after manual review first, machine recognition may no longer be needed.

[0033] In another optional implementation, when the computing power resource returns below the preset threshold, the step of automatically reviewing the extracted pictures in chronological order is resumed; Specifically, the step of resuming the automatic review of the extracted pictures in chronological order includes: Judging the timeliness of the currently extracted picture; If the timeliness of the picture is lower than the preset timeliness, the automatic review of the picture is abandoned, and the automatic review starts from the pictures that meet the preset timeliness; Among them, each extracted picture has a corresponding message body content, and the message body content includes a picture serial number; The judging of the timeliness of the currently extracted picture includes: Judging the timeliness of the currently extracted picture according to the picture serial number corresponding to the currently extracted picture.

[0034] In another optional implementation, after the pictures extracted in the above reverse order are manually reviewed, corresponding marks are attached to the pictures after manual review; The step of resuming the automatic review of the extracted pictures in chronological order includes: Judging whether the currently extracted picture has been manually reviewed. If so, the automatic review step is omitted.

[0035] In this implementation, since the manual review is carried out in the order of the time stamps of the pictures, and the machine automatic review will extract pictures in reverse order of time when its computing power resources are tight and then carry out the review. Therefore, after its review is resumed, it is possible that the pictures currently being machine-reviewed have been manually reviewed. So, first judge whether it has been manually reviewed. If so, directly omit the automatic review step, which can further prevent waste of resources.

[0036] In another optional implementation, as Figure 2An optimized device for picture review is shown, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of an optimized method for picture review described in any of the above embodiments are implemented.

[0037] In another alternative embodiment, a computer-readable storage medium stores computer program instructions. When the computer program instructions are executed by a processor, the steps of an optimized method for picture review described in any of the above embodiments are implemented.

[0038] In summary, for an optimized method, device, and computer-readable storage medium for picture review provided by the present invention, random frame extraction actions are performed on all videos to generate random picture information. All picture recognition tasks are preferentially sent to mq, and mq is used to stack tasks to slow down the situation of burst traffic. At the same time, when the picture recognition program is under high pressure or lacks resources, it is necessary to ensure that relatively new data can be recognized preferentially to ensure that the latest videos can be automatically recognized prior to manual review. At this time, after consuming the messages in mq, the task data will be preferentially sent to the zset data structure of redis, and the score is used to record the timestamp, and then relevant message sorting operations are performed. When there are available processing resources, the latest recognition tasks will be preferentially processed in the order of the score timestamp to ensure that all recognition resources preferentially process relatively new video data, ensure that automatic recognition can be superior to manual review, and prevent waste of invalid resources.

[0039] The above are only the embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent transformation made using the specification and drawings of the present invention, or directly or indirectly applied in the relevant technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. An optimized method for picture review, characterized in that, Including the steps: Automatically review the extracted pictures in chronological order; Monitor the computing power resources for picture review. When the computing power resources exceed the preset threshold, stop the automatic review of the pictures, extract the pictures in the reverse order of the chronological order and then conduct automatic review.

2. The optimized method for picture review according to claim 1, characterized in that, The automatically reviewing the extracted pictures in chronological order includes: Perform a random frame extraction action on the video to be reviewed, generate random frame extraction pictures, and send the frame extraction pictures to the message queue; Extract pictures from the message queue in sequence for automatic review.

3. An optimization method for picture review according to claim 1 or 2, characterized in that, The extracting pictures in the reverse order of the chronological order includes: Write the pictures extracted in chronological order into the zset structure of redis; Extract pictures from the zset structure.

4. The optimized method for picture review according to claim 3, characterized in that, Each extracted picture has a corresponding message body content, and the message body content includes a timestamp; When writing the picture into the zset structure of redis, use the score field to record the timestamp corresponding to the picture; When extracting pictures from the zset structure, extract the pictures in reverse chronological order according to the timestamp recorded in the score field.

5. The optimized method for picture review according to claim 1 or 2, characterized in that, When the computing power resources recover below the preset threshold, resume the steps of automatically reviewing the extracted pictures in chronological order.

6. The optimized method for picture review according to claim 5, wherein, The resuming the steps of automatically reviewing the extracted pictures in chronological order includes: Judge the timeliness of the currently extracted picture; If the timeliness of the picture is lower than the preset timeliness, abandon the automatic review of the picture and start the automatic review from the pictures that meet the preset timeliness.

7. An optimized method for picture review according to claim 6, characterized in that, Each extracted picture has a corresponding message body content, and the message body content includes a picture serial number; The judging the timeliness of the currently extracted picture includes: Judge the timeliness of the currently extracted picture according to the picture serial number corresponding to the currently extracted picture.

8. An optimized method for picture review according to claim 1 or 2, characterized in that, The automatically reviewing the extracted pictures includes: Automatically identify the picture content and text of the extracted pictures, judge whether there is sensitive content, and if so, determine its risk level according to the sensitive content; Set a corresponding risk level label for the picture based on the determined risk level.

9. An optimized device for picture review, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it realizes the steps of an optimized method for picture review as described in any one of claims 1 to 8.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, it realizes the steps of an optimized method for picture review as described in any one of claims 1 to 8.

Images