Image management apparatus, control method of image management apparatus, and storage medium

Abstract

The invention provides an image management apparatus, a control method of the image management apparatus, and a storage medium. An image management apparatus that manages an image includes: a transmission unit configured to transmit an image analysis request to an image analysis apparatus that analyzes an image; a receiving unit configured to receive an analysis result from the image analysis apparatus; a result code determination unit configured to determine a processing result based on a result code included in the analysis result; and a subsequent processing determination unit configured to determine a subsequent processing related to the image analysis based on the determination by the result encoding determination unit.

Description

Technical Field

[0001] The present invention relates to an image management device that manages images sent from a camera device, an image analysis device that analyzes images, a determination and control method for determining subsequent processing based on a result of analysis processing, and a storage medium. Background Art

[0002] As a development method for software systems, there is a development method called "microservice architecture" by which the functions of an application are divided into small "services" that operate in collaboration with the system. Services operated by individual independent processes collaborate through remote communication using a lightweight mechanism such as a RESTful application programming interface (API). For example, even if a service fails, its impact can be suppressed locally, which brings the advantage of improving the fault tolerance of the entire application.

[0003] In recent years, applications that adopt this microservice architecture have become increasingly common. For example, in an image management service that manages images, there is an application that is configured to independently provide functions such as an image analysis function and an image sending function that sends images to an external storage or a social network service (SNS) as separate services.

[0004] In an application configured with the above-mentioned microservice architecture, a service communicates with another service through a RESTful API or the like to make a call. The call between services may fail due to various reasons such as a setting error on the caller side, a network error, or a failure in the called service. If the call to the service fails, the function of the application cannot be provided, causing inconvenience to the user. Therefore, it is desirable to prevent the failure of the call between services as much as possible.

[0005] Japanese Patent Application Publication No. 2022-109916 discusses a technology for increasing the possibility of obtaining printed materials while reducing the burden on users in a printing system having multiple processing paths between a printer and an information processing device. If processing on the first processing path fails, the system switches to the second processing path and retries printing, thereby reducing the number of printing failures.

[0006] However, according to the technology discussed in Japanese Patent Application Publication No. 2022-109916, retry processing is performed every time a processing failure occurs, regardless of the content of the processing failure. Depending on the cause of the failure, there may be a situation where the retry process always fails again. Summary of the invention

[0007] The present invention is proposed to solve the above-mentioned problem, and aims to reduce the failure of retrying an analysis request in an image management device communicating with an image analysis device.

[0008] One embodiment of the present disclosure provides an image management device for managing images, comprising: a sending unit, configured to send an image analysis request to an image analysis device for analyzing the image; a receiving unit, configured to receive an analysis result from the image analysis device; a result code determination unit, configured to determine a processing result based on a result code included in the analysis result; and a subsequent processing determination unit, configured to determine subsequent processing related to the image analysis based on a determination of the result code determination unit.

[0009] Further features will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a diagram showing an overall configuration.

[0011] Figure 2 is a diagram showing the hardware configuration of the image management apparatus.

[0012] Figure 3 is a diagram showing a functional configuration of an image management apparatus.

[0013] Figure 4 is a diagram showing an example of result encoding and its related information.

[0014] Figure 5A and Figure 5B : is a flowchart showing the flow of processing from image transmission processing to processing for storing analysis results.

[0015] Figure 6 is a diagram showing an example of information stored in the analysis image management unit.

[0016] FIG. 7A to FIG. 7D An example of response information of the image analyzing device is shown.

[0017] Figure 8 is a diagram showing an example of information stored in a job management unit. . DETAILED DESCRIPTION

[0018] Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. The configurations described in the following exemplary embodiments are merely examples, and the present invention is not limited to the configurations shown in the drawings.

[0019] In the present exemplary embodiment described below, an example is described in which an image management device that manages images uploaded from an imaging device communicates with an image analysis device that performs analysis processing on the images and determines subsequent processing based on a result code received together with the result of the analysis processing.

[0020] <Overall Configuration>

[0021] Will refer to Figure 1 The overall configuration according to the present exemplary embodiment is described.

[0022] like Figure 1 As shown, the overall configuration includes at least one camera device 101 , an image management device 102 , and an image analysis device 103 . These components are connected via a network 104 .

[0023] The imaging device 101 is an information processing device such as a camera, a mobile phone, a smartphone, or a tablet computer.

[0024] The image management device 102 manages the images transmitted from the imaging device 101 .

[0025] When a request is made to perform image analysis on an image managed by the image management apparatus 102, the image analysis apparatus 103 accepts the request and performs image analysis. After the image analysis is completed, the image analysis apparatus 103 returns the analysis result to the image management apparatus 102.

[0026] The above is the overall configuration of the present exemplary embodiment.

[0027] <Hardware Configuration>

[0028] Will refer to Figure 2 The hardware configuration of the image management apparatus 102 according to the present exemplary embodiment is described.

[0029] like Figure 2 As shown, the image management apparatus 102 includes at least a central processing unit (CPU) 201 , a random access memory (RAM) 202 , a read only memory (ROM) 203 , a network interface 204 , an external storage device 205 , a display device 206 , and an input device 207 .

[0030] The CPU 201 controls the operations of the units constituting the image management apparatus 102 , and functions as a main component for executing various processes (described below) as processes executed by the image management apparatus 102 .

[0031] The RAM 202 is a memory that temporarily stores data and control information, and is used as a work area used by the CPU 201 when executing various processes.

[0032] The ROM 203 stores fixed operation setting values, operation programs, and the like of the image management apparatus 102 .

[0033] The network interface 204 provides a function for connecting to a network to perform communication. The network interface 204 allows the image management apparatus 102 to transmit and receive data to and from an external apparatus.

[0034] The external storage device 205 is a device for storing data, and has an interface for receiving input / output (I / O) commands for reading and writing data. The external storage device 205 may be, for example, a hard disk drive (HDD), a solid state drive (SSD), an optical disk drive, a semiconductor storage device, or other types of storage devices. The external storage device 205 stores computer programs and data for causing the CPU 201 to perform processing (described below) as processing to be performed by the image management apparatus 102.

[0035] The display device 206 is, for example, a liquid crystal display (LCD) or the like, and displays various information to the user.

[0036] The input device 207 is, for example, a keyboard, a mouse, a touch panel, etc., and receives input from the user.

[0037] <Function Configuration>

[0038] Will refer to Figure 3 The configuration of the image management apparatus 102 according to the present exemplary embodiment is described.

[0039] The image management apparatus 102 in this exemplary embodiment has an image management unit 301, a sending unit 302, a receiving unit 303, a result coding determination unit 304, a subsequent processing determination unit 305, a retry control unit 306, an analysis image information management unit 307, a job management unit 308, and an analysis result management unit 309.

[0040] The image management unit 301 stores images. The images sent from the camera 101 are managed by the image management unit 301 .

[0041] The transmission unit 302 transmits a processing request for analyzing the image managed by the image management unit 301 to the image analysis device 103 .

[0042] The receiving unit 303 receives the result of the image analysis process from the image management apparatus 102 .

[0043] The result code determination unit 304 determines whether the process ends normally or abnormally by referring to the result code included in the result of the image analysis process received by the receiving unit 303. If the process ends abnormally, the result code determination unit 304 further determines the cause category.

[0044] The subsequent processing determination unit 305 determines the subsequent processing based on the determination result of the result code determination unit 304 .

[0045] The retry control unit 306 in the subsequent process determination unit 305 determines whether to retry according to the cause category. The retry control unit 306 also determines whether the retry upper limit set in the system has been reached.

[0046] When an image analysis request is issued to the image analyzing device 103 , the analysis image information management unit 307 manages information on an image to be analyzed.

[0047] When an image analysis request is issued to the image analyzing apparatus 103, the job management unit 308 manages the association between the image to be analyzed and the analysis processing to be executed. In the present exemplary embodiment, the analysis processing executed on one image is called a "job", and a job identification (ID) that uniquely identifies the job is assigned each time the analysis processing is executed.

[0048] The analysis result management unit 309 manages the analysis result received from the image analyzing device 103 .

[0049] Next, we will refer to Figure 4 The result code and information associated with the result code in this exemplary embodiment are described. The result code is an identification code, such as A01-002, associated with a failure reason and a failure reason category.

[0050] The analysis result received from the image analyzing device 103 includes a result code 401 .

[0051] The processing result 402 indicates the result of the image analysis processing, and takes a value of "normal end" or "abnormal end".

[0052] The failure reason 403 indicates the failure reason in the image analysis processing, and also indicates the content of the failure reason. If the processing result indicates "normal end", this item has no value.

[0053] The failure cause category 404 indicates a cause category for classifying the failure cause, and takes one value of “transmission side cause”, “reception side cause”, and “communication path side cause”.

[0054] In the failure cause category, the sending side reason indicates the reason for the failure caused by the sending side. The failure reasons related to the sending side include "no required parameters", "incorrect (invalid) request header", "invalid parameters", "request not using the specified method", etc.

[0055] In the failure cause category, the communication path side cause indicates the cause of the failure caused by the communication path side, and the failure causes related to the communication path side include "timeout error".

[0056] In the failure reason category, the receiving side reason indicates the reason for the failure caused by the receiving side. The failure reasons related to the UI receiving side include "failed to download image", "image in unsupported format", "image corruption", "server error", etc.

[0057] <Processing Flow>

[0058] In the present exemplary embodiment, a series of processing steps from when an image is transmitted from the imaging device 101 to the image management device 102 to when the image analysis device 103 performs analysis processing and the image management device 102 stores the analysis result will be described.

[0059] Figure 5A and Figure 5B 1 is a flowchart showing a series of steps from when an image is transmitted from the imaging device 101 to the image management device 102 to when the image analysis device 103 executes analysis processing and the image management device 102 stores the analysis result.

[0060] In step S501, the image captured by the camera 101 is sent to the image management device 102. The processing in this step causes the image to be stored in the image management unit 301. When the image is stored in the image management device 102, an image ID that uniquely identifies the image is assigned to the image. The storage of the image starts the image analysis process. In step S502, information about the image to be analyzed is written in the analysis image information management unit 307. Figure 6 An example of information stored in the analysis image information management unit 307 is shown.

[0061] The stored information includes an image ID 601 to be analyzed, a user ID 602 ​​which is an identifier of a user who owns the image, an image storage location 603 which indicates a storage location of the image, and an image analysis start time 604 which is the time when image analysis starts. The image storage location 603 is, for example, a path on a cloud storage, a path to a storage area in the image management apparatus 102, or the like. The image analysis start time 604 is the time when information is written to the analysis image information management unit 307.

[0062] Next, in step S503, a signed uniform resource location (URL) of the image to be analyzed is published. In the present exemplary embodiment, the signed URL of the image is sent to the image analysis device 103 as a request parameter of the image analysis request. The image analysis device 103 accesses the URL to download the image. The method of sending the image is not limited to the above method. The image data may be sent directly to the image analysis device 103, or the image may be placed in a storage area accessible to the image analysis device 102.

[0063] In step S504, the image analysis device 103 is requested to perform analysis processing. The request includes the signed URL issued in step S503 and the notification destination (endpoint) for receiving the analysis result as request parameters. In the present exemplary embodiment, when an analysis processing request is made to the image analysis device 103, the image analysis device 103 makes two types of responses: a synchronous response and an asynchronous response, and the asynchronous response returns the analysis result after the analysis processing is completed. Each response contains a job ID and a result code. FIG. 7A to FIG. 7D An example of response information from the image analyzing device 103 is shown. FIG. 7A to FIG. 7D The following sections show synchronous response examples for "normal completion" and "abnormal completion", as well as asynchronous response examples for "normal completion" and "abnormal completion".

[0064] In step S505, a synchronization response is received from the image analysis device 103. Then, in step S506, a result code included in the response is determined. In steps S507 and S508, reference is made to Figure 4 The result code in determines whether the image analysis processing is successful (normal end) or failed (abnormal end). If the image analysis processing fails, the subsequent processing is determined based on the failure cause category 404. If the failure cause category is a sending side cause ("Yes" in step S507), the processing proceeds to step S509. If the failure cause category is not a sending side cause ("No" in step S507), the processing proceeds to step S508. If the failure cause category is a receiving side cause or a communication path side cause ("Yes" in step S508), the processing proceeds to step S510. If the failure cause category is not a receiving side cause or a communication path side cause ("No" in step S508), the processing proceeds to step S513.

[0065] In the present exemplary embodiment, in steps S509 and S510, the image stored in the image management apparatus 102 is temporarily copied to another area in the image management apparatus 102. When investigating an image for which the image analysis processing has failed, the image to be investigated may no longer exist due to the user deleting the image from the image management apparatus 102, the user withdrawing from membership, etc. In order to investigate the image for which the analysis has failed even in such a case, if an error occurs in the image analysis processing, the image is temporarily copied to another area. At this time, the processing content of step S509 and the processing content of step S510 may be different. For example, in step S509, a notification of the occurrence of an error may be issued to the administrator of the image management apparatus 102.

[0066] After step S510, a retry process is performed. In the present exemplary embodiment, the upper limit of the number of retries (retry count) for the analysis process of each image is set to 3. If the analysis process fails after 3 retries, the analysis process is determined to be "failed". In step S511, the number of retries is increased by 1, and in step S512, it is determined whether the number of retries has exceeded the upper limit of 3. If it is determined in step S512 that the upper limit has not been exceeded ("No" in step S512), the process returns to step S504. In step S504, the image analysis request is sent to the image analysis device 103 again.

[0067] If the analysis process is determined to be "successful" based on the result code of the synchronization response, then in step S513, the image and information related to the job are written to the job management unit 308. Figure 8 An example of information to be stored in the job management unit 308 is shown.

[0068] This information includes a processing count 803, which indicates the number of times the analysis process is performed using the combination of the job ID 801 and the image ID 802. The initial value of the processing count 803 is "1", and the processing count is incremented each time the retry process is performed. When three retries are made (i.e., the upper limit), the processing count becomes "4".

[0069] After issuing the analysis processing request in step S504, the image analysis device 103 accesses the received signed URL, downloads the image, and starts the analysis processing. When the analysis processing is completed, the image analysis device 103 sends the result of the analysis processing as an asynchronous response to the result notification destination. In step S514, the image management device 102 receives the asynchronous response from the image analysis device 103. In step S515, the job ID included in the received asynchronous response is read from the job management unit 308 to obtain the image information and the number of retries. In step S516, the result code included in the asynchronous response is determined. As in steps S507 and S508, in steps S517 and S518, reference is made to the result code. Figure 4 The result code in determines whether the image analysis processing is successful (normal end) or failed (abnormal end).

[0070] If the result code in the asynchronous response is "normal end" and the analysis process has been "successful" ("No" in steps S517 and S518), then in step S519, the analysis result is stored in the analysis result management unit 309. The received analysis result includes Figure 7C The information shown.

[0071] In the present exemplary embodiment, the number of retries (retry count) for each image is controlled in a series of steps (steps S504 to S518) in the analysis process. For example, assume that the analysis process is started for an image with an image ID of "002" (image ID = 002). If the analysis result is determined to be "failed" based on the result code in the synchronization response in step S508 and a retry is performed, then in step S511, the number of retries becomes "1". If the analysis result is determined to be "successful" based on the result code in the synchronization response after steps S504 to S508 ("No" in step S508), the process proceeds to step S513. In step S513, the process count "2" is written to the job management unit 308 ( Figure 8 If the analysis result is determined to be "failed" based on the result code in the asynchronous response in step S518, and retry is performed again, the number of retries becomes "2" in step S511. If the analysis result is determined to be "successful" based on the result code in the synchronous response again, the processing count "3" is written to the job management unit 308 in step S513. Figure 8 Similarly, if the analysis result is determined to be "failed" based on the result code in the asynchronous response again in step S518, and a retry is performed again, the number of retries becomes "3" in step S511. If the analysis result is determined to be "successful" based on the result code in the synchronous response again, the processing count "4" is written to the job management unit 308 in step S513. Figure 8 Finally, if the analysis result is determined to be "failed" again based on the result code in the asynchronous response in step S518, the number of retries becomes "4" in step S511. In step S512, if it is determined that the number of retries exceeds the upper limit ("Yes" in step S512), the analysis process is determined to be "failed".

[0072] The above is a flow of a series of processing steps from the image capturing device 101 sending an image to the image management device 102, to the image analysis device 103 executing analysis processing, and to the image management device 102 storing the analysis result.

[0073] In the present exemplary embodiment, an example of issuing a request for one analysis process to the image analysis device 103 is shown. Alternatively, multiple calls may be made in a multi-stage manner. In this case, a result code is issued for each result of the first analysis process (e.g., object analysis process) and the second analysis process (e.g., face recognition process) in the same manner, and subsequent processing is determined.

[0074] Thus, in the image management apparatus 102 and the image analysis apparatus 103 according to the present exemplary embodiment, the failure cause category is determined based on the result code received together with the image analysis result, and the subsequent processing is determined. This makes it possible to reduce failures in the analysis processing without unnecessary retries when analyzing an image.

[0075] Other embodiments

[0076] In addition, the embodiments of the present invention can be implemented by reading and executing computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more completely referred to as a "non-transitory computer-readable storage medium") to perform one or more functions in the above-mentioned embodiments, and / or a computer of a system or device including one or more circuits (e.g., an application-specific integrated circuit (ASIC)) for performing one or more functions in the above-mentioned embodiments, and the embodiments of the present invention can be implemented by, for example, reading and executing the computer executable instructions from the storage medium by the computer of the system or device to perform one or more functions in the above-mentioned embodiments, and / or controlling the one or more circuits to perform one or more functions in the above-mentioned embodiments. The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessing unit (MPU)), and may include a network of separate computers or separate processors to read and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or a storage medium. The storage medium may include, for example, a hard disk, a random access memory (RAM), a read-only memory (ROM), a memory of a distributed computing system, an optical disk (such as a compact disc (CD), a digital versatile disc (DVD), or a Blu-ray disc (BD)) TM ), one or more of a flash memory device and a memory card, etc.

[0077] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0078] According to the present invention, it is possible to reduce failures in retrying analysis requests in an image management device that communicates with an image analyzing device.

[0079] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments.The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An image management device for managing images, comprising: a sending unit configured to send an image analysis request to an image analysis device that analyzes the image; a receiving unit, configured to receive an analysis result from the image analyzing device; a result code determining unit configured to determine a processing result based on a result code included in the analysis result; as well as A subsequent processing determination unit is configured to determine subsequent processing related to the image analysis based on the determination of the result encoding determination unit.

2. The image management device according to claim 1, wherein: In the case where the result code determination unit determines that the image analysis has failed, the result code determination unit classifies the failure of the image analysis based on the result code into one of a failure caused by a sending side of the image analysis request, a failure caused by a communication path, and a failure caused by a receiving side of the image analysis request.

3. The image management device according to claim 1, wherein: The subsequent processing determination unit includes a retry control unit configured to determine whether to retry the image analysis request.

4. The image management device according to claim 3, wherein: The retry control unit retries the image analysis request in a case where the analysis failure of the image is classified as a failure caused by a communication path or a failure caused by a receiving side of the image analysis request.

5. The image management device according to claim 3, wherein: The retry control unit does not retry the image analysis request in a case where the analysis failure of the image is classified as a failure caused by the transmission side of the image analysis request.

6. The image management device according to claim 3, wherein: The subsequent processing determination unit determines to make a notification to an administrator as a subsequent processing in a case where the retry control unit determines not to retry the image analysis request.

7. The image management device according to claim 2, wherein: The failure caused by the sending side of the image analysis request is at least one of a request without required parameters, an invalid request header, an incorrect parameter type, an invalid parameter, and not using a specified method.

8. The image management device according to claim 2, wherein: A failure caused by the communication path is a timeout error.

9. The image management device according to claim 2, wherein: The failure caused by the receiving side of the image analysis request is at least one of a failure to download an image, a failure to upload a result, an image in an unsupported format, an image corruption, and a server error.

10. The image management device according to claim 1, wherein: The result code is an identification code of A01-002 and is associated with a failure cause or a failure cause category.

11. The image management device according to claim 1, wherein: In a case where the result code determination unit determines that the image analysis has failed, the subsequent process determination unit determines to copy the image as the subsequent process.

12. A control method for an image management device for managing images, the control method comprising: sending an image analysis request to an image analysis device that analyzes the image; receiving an analysis result from the image analyzing device; determining a processing result based on a result code included in the analysis result; as well as Subsequent processing related to the image analysis is determined based on the determination in the determining step.

13. A non-transitory storage medium storing a program for causing an image management device to execute a control method, the control method comprising: sending an image analysis request to an image analysis device that analyzes the image; receiving an analysis result from the image analysis device; determining a processing result based on a result code included in the analysis result; as well as Subsequent processing related to the image analysis is determined based on the determination in the determining step.

Images