Object recognition system and object recognition program

Abstract

This invention provides an object recognition system and program that minimize computational complexity while maintaining accuracy. [Solution] The object recognition system comprises: object detection means for detecting two or more parts of an object to be detected that is captured in a frame image input from a camera; fitness calculation means for calculating the fitness of the object to be detected as a recognition target based on the position and size of the two or more parts; comparison means for comparing the fitness of the object to be detected with a predetermined reference value; and object recognition means for recognizing only the objects to be detected that meet the reference value as a result of the comparison.

Description

Technical Field

[0001] The present invention relates to an object recognition system and an object recognition program that can reduce the amount of calculation as much as possible while maintaining accuracy.

Background Art

[0002] An object recognition system is used to detect an object such as a person shown in an image captured by a mounted camera and output the result of recognizing the attributes of the detected object. In such an object recognition system, a trained model (hereinafter referred to as "learning model") using a neural network (hereinafter referred to as "NN: Neural Network") trained to output the detection result of an object shown in the input target image data, and a learning model using an NN trained to output the attributes of the detected object are used.

[0003] With the improvement of the computing power of processors and the technological improvement of hardware, instead of a configuration that collects data and performs image processing on a server with abundant computing resources, processing using a learning model is possible even on an edge device with relatively scarce computing resources.

[0004] When deploying an object recognition system with pre-customized conditions etc. for various environments, since it takes man-hours to confirm the condition settings, Patent Document 1 discloses a system that can remotely adjust parameters such as the angle of view of an image captured by a camera in an object recognition system.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] Even on servers with abundant computing resources, there is a need for accurate detection and recognition with lighter processing. Furthermore, there is a need to achieve object detection and recognition processing that is both lightweight and accurate, enabling precise real-time processing even on edge devices with relatively limited computing resources.

[0007] The present invention aims to provide an object recognition system and an object recognition program that can reduce the amount of computation as much as possible while maintaining accuracy. [Means for solving the problem]

[0008] An object recognition system according to one embodiment of the present disclosure includes: object detection means for detecting two or more parts of an object to be detected that are captured in a frame image input from a camera; fitness calculation means for calculating the fitness of the object to be detected as a recognition target based on the position and size of the two or more parts; comparison means for comparing the fitness as a recognition target with a predetermined reference value; and object recognition means for recognizing only the objects to be detected that meet the reference value as a result of the comparison.

[0009] The object recognition system disclosed herein determines whether the object has a high degree of relevance as a target for recognition by determining whether the portion of the image that can be used to calculate feature quantities that can identify the attributes of the object to be detected is sufficiently or clearly visible in the image. The object recognition system acquires video captured by a camera as time-series frame images and performs recognition processing only on the frame images that are judged to have a high degree of relevance from the frame images that can be acquired sequentially by the above-described process. Rather than allocating computational resources to low-accuracy recognition processing targeting frame images in which the characteristic parts are unclear or hidden, it is possible to reduce the load on computational resources and improve accuracy by performing recognition only on frame images in which features can be clearly captured.

[0010] If the position and size of two or more parts that are determined to be the target of detection are appropriate, the frame image can be recognized with high accuracy. The appropriate position and size of the two or more parts will vary depending on what is to be detected and what is to be recognized. By making it possible to change the settings for the two or more parts and the conditions for whether or not it is suitable as a target for recognition depending on the target, high accuracy can be maintained for a variety of targets.

[0011] Furthermore, if the detection target is a person, the two or more parts may include the head portion or the entire outline of the standing figure used for person detection, and the face portion used for recognizing attributes such as age or gender. If the detection target is a person, the two or more parts may also include parts that should show the clothing or accessories worn, or the items being carried. If the detection target is, for example, a vehicle, the two or more parts may include the entire vehicle used for vehicle detection, and the front or rear portion used for recognizing attributes such as vehicle type or color. If the detection target is a retail item, the two or more parts may include the entire item used for item detection, and characteristic parts of the packaging or characteristic parts of the item used for identifying the retail item or recognizing its color. The two or more parts may be set and changed as appropriate depending on what the detection target is and what features should be recognized for the detection target.

[0012] In an object recognition system according to one embodiment of the present disclosure, the object detection means may detect two or more parts of the object to be detected by determining the bounding boxes of two or more parts of the object to be detected, and the fitness calculation means may calculate the fitness of the object to be detected as a recognition target based on the positions and sizes of the bounding boxes of the two or more parts determined by the object detection means.

[0013] In the object recognition system of this disclosure, the bounding box used for detecting an object, including a person, may be treated as a range corresponding to two or more parts. The object recognition system can confirm that there are sufficiently or clearly visible parts in the frame image from which feature quantities that can identify the attributes of the object to be detected can be calculated based on the positional relationship and size ratio of the bounding boxes corresponding to the two or more parts.

[0014] In one embodiment of the object recognition system of this disclosure, the object detection means detects two rectangular portions of the object to be detected, wherein one rectangular portion encompasses the other rectangular portion, and the fitness calculation means determines the distance between a predetermined vertex in one of the rectangles and a corresponding vertex in the other rectangle, and calculates the fitness of the object to be detected as a recognition target based on this distance.

[0015] The object recognition system of this disclosure may use rectangles that indicate the area in which an object is captured, which are commonly used in object detection techniques. The object recognition system can confirm that a portion of the frame image is sufficiently or clearly visible from the positional and sizing relationship between the rectangles corresponding to each of the two portions, from which feature quantities that can identify the attributes of the object to be detected can be calculated. The object recognition system can confirm that the rectangles are clearly visible if, for example, the rectangles corresponding to the head and face of a person are one (head) encompassing the other (face), and the other (face) not being partially contained within the range of the other. The positional and sizing relationship between one and the other will differ depending on the object to be detected and the object to be recognized.

[0016] In one embodiment of the object recognition system of this disclosure, the object detection means and the object recognition means are a trained object detection model and a trained object recognition model, and the system may further include a modification means that changes the trained object detection model, the trained object recognition model and the reference value in response to instructions from an operator via the cloud.

[0017] The object recognition system disclosed herein allows for remote modification of either the learning model or the reference value to ensure appropriate processing depending on the installation environment of the detection target, recognition target, or image captured by the camera. The object recognition system disclosed herein reduces the hassle and effort required to configure settings at the camera installation site each time at least one of the detection target or recognition target changes, while enabling the selection and modification of the learning model or reference value according to the required accuracy. The object recognition system disclosed herein can be remotely adjusted via the cloud to perform lighter and more accurate processing.

[0018] In one embodiment of the object recognition system of this disclosure, the object recognition means is a vision language model, and may further include text modification means that changes the input text to the vision language model in response to instructions from an operator via the cloud, and recognition processing modification means that changes the content of the object recognition processing performed by the vision language model in response to the input text.

[0019] The object recognition system disclosed herein employs a vision language model for object recognition, thereby enabling the recognition content to be changed based on the input text. By employing a vision language model, the recognition target can be changed with a single recognition model, simplifying the configuration.

[0020] In one embodiment of the object recognition system of this disclosure, the degree of suitability of the object to be detected as a recognition target may be used as the degree of confidence of the recognition result of the object to be detected.

[0021] In the object recognition system of the present disclosure, for a frame image in which the way the characteristic part of the detection target appears is unclear or hidden, etc., and the detection target cannot be photographed to the extent that its attributes can be recognized, the fitness is calculated low. Conversely, for a frame image in which the way the characteristic part of the detection target appears is clear and the detection target can be photographed to the extent that its attributes can be recognized, the fitness is calculated high. By outputting the fitness for the frame image, the reliability level of the recognition result can be specified, which is useful when using the recognition result.

[0022] The object recognition system according to an embodiment of the present disclosure may use the reliability score of the object recognition result by the vision language model and the fitness of the object to be detected as the recognition target as the reliability of the recognition result of the object to be detected.

[0023] In the object recognition system of the present disclosure, the vision language model outputs a reliability score of the recognition result. For a frame image that cannot be accurately recognized, the vision language model outputs a low reliability score of the object recognition result. Conversely, for a frame image that can be accurately recognized, the vision language model outputs a high reliability score of the object recognition result. By outputting the reliability score of the object recognition result by the vision language model for each frame image, the reliability level of the recognition can be specified, which is useful when using the recognition result. The fitness is low when the way the characteristic part of the detection target appears is unclear or hidden, etc., and the detection target cannot be photographed to the extent that its attributes can be recognized. By using these reliability scores and the fitness as the reliability of the recognition result of the object to be detected, an accurate reliability can be output.

[0024] In the object recognition system according to an embodiment of the present disclosure, the changing means may change the learned object detection model and the learned object recognition model by replacing only the task head part without replacing the backbone part that extracts the feature amount of the frame image for both the learned object detection model and the learned object recognition model.

[0025] In the object recognition system of the present disclosure, for the part that extracts feature amounts from frame images, since it is often common processing for various detection targets and recognition targets, even if the detection target or recognition target changes, the backbone part is not changed, and only the task head part can be replaced. Thus, depending on what the detection target is and what the recognition target corresponding to the detection target is, a recognition system that conforms to various conditions can be realized by replacing only the necessary parts as much as possible without replacing everything.

[0026] In an object recognition system according to an embodiment of the present disclosure, there are provided a head detection means for detecting a person's head captured in a frame image input from a camera, a face detection means for detecting a person's face captured in the frame image, a face orientation detection means for detecting the face orientation of the face detected by the face detection means, and a fitness calculation means for calculating the fitness of the face of the detection target as a face authentication target based on the positions and sizes of the head and face and the face orientation detected by the face orientation detection means, a comparison means for comparing the fitness as the face authentication target with a predetermined reference value, and a face authentication means for performing face authentication processing on the face detected by the face detection means. The face authentication means performs face authentication processing only on the face of the detection target that has cleared the reference value as a result of the comparison by the comparison means.

[0027] An object recognition program according to an embodiment of the present disclosure causes a computer to function as an object detection means for detecting two or more parts of an object of a detection target captured in a frame image input from a camera, a fitness calculation means for calculating the fitness of the object of the detection target as a recognition target based on the positions and sizes of the two or more parts, a comparison means for comparing the fitness as the recognition target with a predetermined reference value, and an object recognition means for recognizing only the object of the detection target that has cleared the reference value as a result of the comparison.

[0028] An object recognition program according to one embodiment of the present disclosure is an object recognition program for which a computer functions as a head detection means for detecting a person's head captured in a frame image input from a camera; a face detection means for detecting a person's face captured in the frame image; a face orientation detection means for detecting the face orientation of the face detected by the face detection means; a suitability calculation means for calculating the suitability of the detected face as a face recognition target based on the position and size of the head and face, as well as the face orientation detected by the face orientation detection means; a comparison means for comparing the suitability as a face recognition target with a predetermined reference value; and a face recognition means for performing face recognition processing on the face detected by the face detection means, wherein the face recognition means performs face recognition processing only on the detected faces that clear the reference value as a result of the comparison by the comparison means. [Effects of the Invention]

[0029] According to this disclosure, by performing recognition processing only on frame images from the camera in which the detection target and the recognition target are clearly depicted to facilitate recognition processing of characteristic parts, the computational load is reduced compared to all frame images, which would require a lot of computing resources, while maintaining a high level of recognition accuracy. [Brief explanation of the drawing]

[0030] [Figure 1] This is a schematic diagram of the object recognition system according to the first embodiment. [Figure 2] This is a block diagram showing the configuration of an edge device. [Figure 3] This is a block diagram showing the configuration of a cloud server. [Figure 4] The client's composition is defined in block. [Figure 5] This flowchart shows an example of the image recognition processing procedure on an edge device. [Figure 6] This is a diagram illustrating processing performed by edge devices. [Figure 7]This is a schematic diagram of the object recognition system according to the second embodiment. [Figure 8] This flowchart shows an example of the model setting process procedure in the object recognition system of the second embodiment. [Figure 9] This flowchart shows an example of the model setting process procedure in the object recognition system of the third embodiment. [Figure 10] This is an explanatory diagram of the processing performed by the edge device in the third embodiment. [Figure 11] This flowchart shows an example of the model setting process procedure in the object recognition system of the fourth embodiment. [Figure 12] This is an explanatory diagram of the processing performed by the edge device according to the fourth embodiment. [Modes for carrying out the invention]

[0031] This disclosure will be described in detail with reference to drawings illustrating its embodiments. The following embodiments describe the object recognition system of this disclosure.

[0032] [First Embodiment] Figure 1 is a schematic diagram of the object recognition system 100 according to the first embodiment. The object recognition system 100 includes a camera 2, an edge device 1 connected to the camera 2, a cloud server 3 that can communicate with the edge device 1 via a network N, and a client 4 that can communicate with the cloud server 3.

[0033] Edge device 1 uses a neural network-based learning model to extract features from image data acquired from camera 2, detect objects such as people based on these features, recognize the attributes of the objects in the image data based on these features, and output the recognition results. Edge device 1 outputs the recognition results of the detected object attributes as text. In the following description, edge device 1 is described as one computer for one camera 2, but it may also be configured with multiple computers for one camera 2, with each computer handling the processing, or one or more computers may be used to perform processing for multiple cameras 2.

[0034] Camera 2 outputs image data using an image sensor that supports visible light and / or near-infrared light. Camera 2 outputs frame image data in a time series at a rate of several fps to tens of fps.

[0035] Edge device 1 and camera 2 can communicate via signal lines or via wireless or wired communication media. For example, edge device 1 and camera 2 can communicate via coaxial cable, USB (Universal Serial Bus), serial bus, wired LAN, wireless LAN, or Bluetooth®.

[0036] Cloud server 3 connects to edge device 1 via network N and functions as a cloud manager that instructs edge device 1 on the processing to be performed. Cloud server 3 functions as a cloud manager for edge device 1 connected to cameras 2, which are installed in different spaces. Cloud server 3 performs manager functions that instruct edge device 1 on the processing to be performed, such as setting reference values ​​that will be referenced in the processing performed by edge device 1 as described later.

[0037] Cloud server 3 acquires the results (text) of the recognition process performed by edge device 1 in each space and stores them in database 300 (see Figure 3). Cloud server 3 may also perform analysis processing such as aggregation processing or statistical processing of attributes related to detected objects for each space and store them in database 300. The recognition processing results stored in cloud server 3 can be viewed by an operator using client 4 by specifying data that identifies the space or data that identifies edge device 1.

[0038] Network N is a wired or wireless communication network that may include a public communication network, a dedicated line, or a carrier network.

[0039] In the object recognition system 100 configured in this way, in order to reduce the processing load on the edge device 1 while maintaining high detection and recognition accuracy on the edge device 1, recognition processing is omitted for frame images where recognition accuracy is likely to be low. Furthermore, the object recognition system 100 accepts the setting of reference values ​​from the cloud server 3 in order to identify frame images that are likely to have low recognition accuracy.

[0040] The details of this object recognition system 100 will be described below.

[0041] Figure 2 is a block diagram showing the configuration of edge device 1. Edge device 1 uses an edge computer. In the following description, edge device 1 comprises a processing unit 10, a storage unit 11, a first communication unit 12, and a second communication unit 13.

[0042] The processing unit 10 includes one or more processors such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), and NPU (Neural Processing Unit). The processing unit 10 includes memory, which is a temporary storage medium such as SRAM (Static Random Access Memory) and DRAM (Dynamic Random Access Memory). The processing unit 10 includes a timer and can obtain time information at each point in time from data from the timer. The processing unit 10 may be configured as a single hardware (SoC: System On a Chip) integrating the processor, memory, storage unit 11, first communication unit 12, and second communication unit 13.

[0043] The processing unit 10 causes the processor to perform image processing based on the image recognition program P1 stored in the memory unit 11 and the learning model deployed from the cloud server 3. The processing unit 10 and the image recognition program P1 mainly correspond to the "fitness calculation means" and "comparison means" in the claim.

[0044] The storage unit 11 is a relatively large-capacity non-temporary storage medium such as a hard disk or flash memory. A portion of the storage unit 11 may be removable.

[0045] The storage unit 11 stores the program (program product) necessary for the processing unit 10 to execute processing, the results of the processing by the processing unit 10, and reference configuration data. The configuration data includes identification data for the device itself. The program product includes the OS (Operating System) program, the image recognition program P1 that runs on the OS, the learning model group M1, and configuration data. The learning model group M1 will be described in detail later.

[0046] The image recognition program P1 stored in the memory unit 11 may be an image recognition program P9 stored on a computer-readable storage medium 9 that the processing unit 10 reads and stores in the memory unit 11, or it may be a program that is pre-stored at the time of shipment. The image recognition program P1 stored in the memory unit 11 may also be an image recognition program P1 that the processing unit 10 downloads from the cloud server 3 or another download server via the second communication unit 13 and stores in the memory unit 11.

[0047] The learning model group M1 stored in the memory unit 11 includes detection models that have been trained to detect, based on the features obtained from the image, whether or not an object is present in the input image, and, if so, the extent to which that object is present in the image. The detection models differ depending on the target, such as a person detection model that detects whether or not a person is present in the image, or a vehicle detection model that detects whether or not a vehicle is present in the image. The detection models included in the learning model group M1 are selected according to the target to be detected.

[0048] The learning model group M1 includes two or more detection models that detect two or more parts of a person or object to be detected. These two or more detection models, the processing unit 10, and the image recognition program P1 correspond to the "object detection means" in the claim. When the object to be detected is a person, the learning model group M1 includes, for example, a head detection model that detects the head and a face detection model that detects the face. The head detection model, the processing unit 10, and the image recognition program P1 correspond to the "head detection means" in the claim, and the face detection model, the processing unit 10, and the image recognition program P1 correspond to the "face detection means" in the claim. When the object to be detected is a person, in another example, the learning model group M1 includes a person detection model that detects the entire person and a foot detection model that detects the feet. When the object to be detected is a vehicle, the learning model group M1 includes, for example, a vehicle detection model that detects the entire vehicle and a license plate detection model that detects the license plate. If the target of detection is a vehicle, the learning model group M1 may include a vehicle detection model that detects the entire vehicle body and a detection model that detects the front door or rear door portion where the vehicle's brand logo is attached. In addition, the detection portion will differ depending on the target of detection.

[0049] The learning model group M1 stored in the memory unit 11 includes a recognition model that recognizes the attributes of a detected person or object. This recognition model, the processing unit 10, and the image recognition program P1 correspond to the "object recognition means" in the claim. The learning model group M1 includes attribute-specific recognition models that recognize attributes such as a person's gender and age, respectively. The learning model group M1 also includes a face orientation detection model that detects the orientation of the face detected by the face detection model described above. This face orientation detection model, the processing unit 10, and the image recognition program P1 correspond to the "face orientation detection means" in the claim. Furthermore, the learning model group M1 includes a face recognition model that performs face recognition processing on the face (image) detected by the face detection model. This face recognition model, the processing unit 10, and the image recognition program P1 correspond to the "face recognition means" in the claim. The recognition models included in the learning model group M1 are selected and stored according to the recognition target. The learning model group M1 may include object-specific models that recognize clothing and accessories worn by detected individuals, or it may include models that recognize the color or pattern of detected objects.

[0050] The learning model group M1 stored in the memory unit 11 may be selected or set by the client 4 via the cloud server 3, selected by the functions of the cloud server 3, or automatically selected by the processing unit 10.

[0051] The memory unit 11 stores configuration data corresponding to the selected group of learning models M1 and the installation environment of the camera 2. The configuration data includes setting information for each of the learning model groups M1, such as the size of the detection target area or the recognition target area in the image. The configuration data stored in the memory unit 11 is selected according to the learning model group M1.

[0052] The configuration data stored in the memory unit 11 includes reference values ​​that will be referenced in the processing procedure described later. The reference values ​​may be initial values ​​or values ​​that have been changed by the client 4 via the cloud server 3, as described later.

[0053] The first communication unit 12 is a device for connecting to the camera 2. The first communication unit 12 may be an interface such as USB (Universal Serial Bus) connected to the camera 2, or it may be a coaxial cable or other serial bus interface. The first communication unit 12 may be a LAN network card or a CAN communication device. The first communication unit 12 may be a communication device that supports wireless networks such as WiFi or Bluetooth (registered trademark). The first communication unit 12 may include multiple communication devices that support various types of cameras 2. The first communication unit 12 may be the same device as the second communication unit 13.

[0054] The second communication unit 13 is a communication device that enables communication with the cloud server 3 via the network N. The second communication unit 13 may be a wired LAN network card, a communication device that enables carrier communication via a carrier network, or a communication device that supports wireless networks such as WiFi or Bluetooth (registered trademark). The second communication unit 13 may support encrypted communication such as SSL with the cloud server 3. The second communication unit 13 may also be an interface for enabling connection with the cloud server 3 via a dedicated line.

[0055] Figure 3 is a block diagram showing the configuration of the cloud server 3. The cloud server 3 is configured to distribute processing across multiple server computers that are connected to each other. The cloud server 3 comprises a processing unit 30, a storage unit 31, and a communication unit 32. The cloud server 3 may consist of a single server computer, provided that it can be connected to by communication from the edge device 1 and the client 4 via the network N.

[0056] The processing unit 30 includes one or more processors such as CPUs, MPUs, GPUs, or NPUs. The processing unit 30 includes memory, which is a temporary storage medium such as SRAM or DRAM. Primarily, the processing unit 30 corresponds to the "modification means" in the claim. Also, primarily, the processing unit 30 corresponds to the "text modification means" in the claim.

[0057] The storage unit 31 is a relatively large-capacity non-temporary storage medium such as a hard disk or flash memory. The storage unit 31 stores the program (program product) and configuration data necessary for the processing unit 30 to execute processing.

[0058] The program product stored in the memory unit 31 includes the server program P3. The server program P3 includes a module that enables it to function as a web server, and can accept data input on the web page displayed to the client 4 and display the calculated data on the web page.

[0059] The server program P3 may be obtained by the processing unit 30 reading the server program P8 stored on a storage medium 8 that can be read from a computer and storing it in the storage unit 31, or it may be obtained by the processing unit 30 downloading it from another download server via the communication unit 32 and storing it in the storage unit 31.

[0060] The communication unit 32 is a communication device that enables communication connections with the client 4 and the edge device 1 via the network N.

[0061] Figure 4 is a block diagram showing the configuration of Client 4. Client 4 is a personal computer, smartphone, or tablet device. Client 4 may be used by the administrator of the space where Camera 2 is installed, or by the operator of the management company of Cloud Server 3.

[0062] Client 4 comprises a processing unit 40, a storage unit 41, a communication unit 42, a display unit 43, and an operation unit 44. The processing unit 40 includes one or more processors such as CPUs, MPUs, GPUs, or NPUs. The processing unit 40 also includes memory, which is a temporary storage medium such as SRAM or DRAM.

[0063] The storage unit 41 is a memory of a non-temporary storage medium such as a hard disk or flash memory. The storage unit 41 stores a client program P4 for the Web server provided by the cloud server 3. The client program P4 is, for example, a Web browser program. The client program P4 may also be a program that causes the processing unit 40 to execute the process of displaying the data provided by the cloud server 3 on the screen.

[0064] The communication unit 42 is a communication device that enables communication with the cloud server 3 via the network N. The communication unit 42 may also be a communication device that enables communication with the cloud server 3 via a dedicated line. The communication unit 42 may also be a communication device that enables direct communication with the second communication unit 13 of the edge device 1 via a wireless communication medium or a USB cable, etc.

[0065] The display unit 43 uses a display such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 43 displays a web page containing text or images based on processing by the client program P4 of the processing unit 40. The display unit 43 may also use a display with a built-in touch panel.

[0066] The operation unit 44 is a user interface such as a keyboard or pointing device that accepts operations from the operator. The operation unit 44 may be a touch panel built into the display of the display unit 43, or it may be physical buttons. The operation unit 44 may be a voice input unit that accepts operations by voice using a voice recognition function. The operation unit 44 can notify the processing unit 40 of the operation information from the operator.

[0067] In the object recognition system 100 configured in this way, the processing procedure for the edge device 1 to perform object recognition by focusing on frame images from the frame images captured by the camera 2 that clearly capture features and allow for object recognition will be described. Figure 5 is a flowchart showing an example of the image recognition processing procedure in the edge device 1. The processing unit 10 of the edge device 1 receives frame images from the camera 2 in chronological order and performs the following processing each time a frame image is received.

[0068] The processing unit 10 acquires a frame image (step S101), inputs the acquired frame image to a first detection model corresponding to the object to be detected (step S102), and obtains a first detection result (step S103). The processing unit 10 inputs the acquired frame image to a second detection model (step S104), and obtains a second detection result (step S105). In step S104, the processing unit 10 may extract the range of the object detected in the first detection result from the frame image and input it to the second detection model.

[0069] The processing unit 10 calculates the degree of relevance of the detected object as a recognition target based on the position and size of the first part of the object obtained as the first detection result and the position and size of the second part of the object obtained as the second detection result (step S106).

[0070] In step S106, if the first part encompasses the second part, the processing unit 10 calculates the distance between a specific position in the first part and a specific position in the second part. The processing unit 10 uses the calculated distance as the degree of fit. The processing unit 10 may also calculate the degree of fit from the proportion of the first part to the extent of the second part. The processing unit 10 may also calculate the degree of fit from the ratio of the length of a specific part of the first part to the length of a specific part of the second part, or it may use the distance between the center position (centroid position) of the first part and the center position (centroid position) of the second part as the degree of fit.

[0071] The processing unit 10 compares the degree of fit calculated in step S106 with a predetermined reference value and determines, based on the comparison, whether the degree of fit meets the conditions using the reference value (step S107). In step S107, the processing unit 10 determines whether conditions such as whether the distance is less than or equal to a predetermined reference value, whether the distance is greater than or equal to a predetermined reference value, or whether the distance is within the range of a predetermined reference value have been met. The processing unit 10 may also determine whether the conditions have been met based on whether the ratio is greater than or equal to a predetermined ratio, less than or equal to a predetermined ratio, or within a predetermined ratio range. The processing unit 10 may also determine whether the conditions have been met based on whether the ratio is greater than or equal to a predetermined rate, whether the ratio is less than or equal to a predetermined rate, or whether the ratio is within a predetermined range. In this claim, "meeting the reference value" means that in step S107, "the degree of fit meets the conditions using the reference value."

[0072] If the processing unit 10 determines that the degree of fit meets the criteria using the reference value (S107: YES), it inputs the frame image acquired in step S101 into the recognition model from the learning model group M1 (step S108). In step S108, the processing unit 10 may also input a partial image obtained by extracting a first part of the first detection result from the frame image, or a partial image obtained by extracting a second part of the second detection result, into the recognition model.

[0073] The processing unit 10 obtains the recognition result from the recognition model (step S109). The processing unit 10 stores the obtained recognition result and the degree of fit calculated in step S106 in association with the identification data of the frame image (step S110), and terminates the process. If there are multiple recognition targets (whose degree of fit meets the condition using the reference value), the processing unit 10 executes the processes in steps S108-S110 according to the number of recognition targets.

[0074] In step S107, if it is determined that the degree of fit does not meet the conditions using the reference value (S107: NO), the processing unit 10 stores the degree of fit calculated in step S106 in association with the identification data of the frame image (step S111), and terminates the process. In this case, the processing unit 10 omits the processing using the recognition model for the frame image acquired in step S101.

[0075] When the recognition results of each frame image stored in the memory unit 11 have been accumulated for a predetermined period or for a predetermined number of frame images, the processing unit 10 of the edge device 1 associates the device's identification data (for identifying the target space) with the frame image identification data and transmits the recognition results and degree of fit data to the cloud server 3. This allows the operator to access the cloud server 3 using the client 4 and refer to the recognition results and degree of fit on the edge device 1 for each space. The degree of fit (as a recognition target for the detected object) can be used as the confidence level of the recognition result for the detected object.

[0076] Furthermore, instead of storing the frame image identification data in the memory unit 11 of the edge device 1 and sending it to the cloud server 3, as described above, the frame image itself may be stored (saved) in the memory unit 11 of the edge device 1 or sent to the cloud server 3, associating it with the recognition result and fitness data. This makes it possible to obtain training images (or images for fine-tuning) for the recognition model used in step S108.

[0077] The processing procedure shown in Figure 5 will be explained with a specific example. Figure 6 is an explanatory diagram of the processing by edge device 1. In the example in Figure 6, edge device 1 uses a head detection model M11 to detect the head of a person and a face detection model M12 to detect the face region, for the purpose of recognizing the age or gender of a person. Edge device 1 uses, for example, an age recognition model M13 to recognize age. The processing unit 10 calculates the degree of fit from the position and size of the head obtained by inputting the frame image to the head detection model M11, and the position and size of the face obtained by inputting the frame image to the face detection model M12. In the example shown in Figure 6, the degree of fit is calculated using the distances D1 and D2 from predetermined vertices of the rectangle detected as the head area (upper left and lower right in Figure 6) to predetermined vertices of the rectangle detected as the face area (upper left and lower right in Figure 6). If the degree of fit (distance D1, D2) meets the criteria using a standard value, the processing unit 10 inputs the first or second portion of the frame image to the age recognition model M13 and stores the recognition result (age and confidence score) from the age recognition model M13 along with the degree of fit. If the degree of fit (distance D1, D2) does not meet the criteria value, the processing unit 10 does not continue processing the frame image, stores the degree of fit for the identification data of the frame image, and terminates processing.

[0078] The lower part of Figure 6 shows a specific example of how the goodness of fit is calculated. Figure 6 shows examples of detection results for Case 1 to Case 3. In Case 1, the processing unit 10 determines that the head area detected from the frame image includes the face area, and furthermore, that the distance D1 between the upper left vertex of rectangle R1 detected as the head area and the upper left vertex of rectangle R2 detected as the face area is less than the first threshold of the reference value, and that the distance D2 between the lower right vertex of rectangle R1 corresponding to the head and the lower right vertex of rectangle R2 corresponding to the face area is less than the second threshold of the reference value. As a result, the processing unit 10 determines that the distances D1 and D2 calculated as the goodness of fit are smaller than the first and second thresholds of the reference value, respectively, and that the conditions are met. In Case 1, the processing unit 10 inputs the target frame image (either the whole or a part of rectangle R1 or R2) into the age recognition model M13 to obtain the recognition result. The processing unit 10 may store and output the confidence score (score) corresponding to the accuracy included in the recognition result output from the age recognition model M13 as the confidence level of the object recognition system 100 for the frame image (the confidence level of the recognition result of the object to be detected included in the frame image). Alternatively, the confidence score (of the object recognition result) and the degree of fit of the object to be detected as a recognition target may be stored and output as the confidence level of the object recognition system 100 for the frame image (the confidence level of the recognition result of the object to be detected included in the frame image).

[0079] In Case 2 of the example shown in Figure 6, the processing unit 10 obtains a rectangle R1 corresponding to the head and a rectangle R2 corresponding to the face from the frame image as detection results, similar to Case 1. In Case 2, the processing unit 10 determines that although the rectangle R1 of the head encompasses the rectangle R2 of the face, the distance D1 between the top-left vertex of the rectangle R1 of the head and the top-left vertex of the rectangle R2 of the face is greater than or equal to the first threshold included in the reference value, and therefore the condition is not met. In Case 2, the processing unit 10 terminates processing without inputting the target frame image to the age recognition model M13, that is, without performing age recognition on the target frame image. The processing unit 10 may store the calculated goodness of fit (distance D1 or distance D2) as the confidence level of the object recognition system 100 for the frame image (confidence level of the recognition result of the object to be detected shown in the frame image) and output it. In this case, the larger the distance D1 or distance D2 used as the goodness of fit, the lower the confidence level will be output.

[0080] In Case 3 of the example shown in Figure 6, the processing unit 10 obtains a rectangle R1 corresponding to the head and a rectangle R2 corresponding to the face from the frame image as detection results, similar to Case 1. In Case 3, the processing unit 10 determines that although the rectangle R1 of the head encompasses the rectangle R2 of the face, the distance D2 between the lower right vertex of the rectangle R1 of the head and the lower right vertex of the rectangle R2 of the face is greater than or equal to the second threshold included in the reference value, and therefore the condition is not met. In Case 3, the processing unit 10 terminates processing without inputting the target frame image to the age recognition model M13, that is, without performing age recognition on the target frame image. The processing unit 10 may store the calculated goodness of fit (distance D1 or distance D2) as the confidence level of the object recognition system 100 for the frame image (confidence level of the recognition result of the object to be detected contained in the frame image) and output it. In this case as well, the larger the distance D1 or distance D2 used as the goodness of fit, the lower the confidence level will be output.

[0081] As shown in Figure 6, by determining whether to proceed with recognition processing based on the conditions that the rectangle R1 representing the head encloses the rectangle R2 representing the face, and the distance between the vertices of the rectangle R1 representing the head and the vertices of the rectangle R2 representing the face is less than a certain threshold, recognition processing can be limited to frame images in which the face is clearly visible enough to sufficiently calculate the facial features. Rather than allocating computational resources to low-accuracy recognition processing for frame images in which characteristic parts are unclear or hidden, it is possible to reduce the load on computational resources and improve accuracy by focusing recognition on frame images in which features can be clearly captured.

[0082] In the example shown in Figure 6, the processing unit 10 uses the distances D1, D2, etc., between the vertices of rectangle R1 corresponding to the head and the vertices of rectangle R2 corresponding to the face as a degree of fit for comparison with a reference value. However, the degree of fit may be calculated by other methods. The degree of fit is not limited to the distance between the vertices of rectangle R1 and rectangle R2; it may also be calculated from the ratio of the area occupied by rectangle R2 of the face to the area occupied by rectangle R1 of the head. The processing unit 10 may also calculate the degree of fit from the ratio of the length of the long side of rectangle R1 of the head to the length of the long side of rectangle R2 of the face. The distance between the center position (centroid) of rectangle R1 and the center position (centroid) of rectangle R2 may also be used as the degree of fit for comparison with a reference value. In this case, the shorter the distance between the center positions, the higher the degree of fit as a recognition target is judged to be.

[0083] In the example shown in Figure 6, the processing unit 10 used the distances D1 and D2 between the vertices of rectangle R1 corresponding to the head and the vertices of rectangle R2 corresponding to the face as the degree of fit for comparison with a reference value. However, if the recognition model is not the age recognition model M13 as described above, but a face recognition model (a model that determines whether the detected face is the same as any of the faces stored (registered) in the memory unit 11, etc.), then in addition to the distances D1 and D2 described above, the processing unit 10 may also use the face orientation score obtained using a face orientation detection model (a score indicating the degree of certainty that the face orientation obtained by inputting the face image, which has had the face region extracted by the face detection model M12 detected, into the face orientation detection model is a face orientation suitable for face recognition) as the degree of fit for comparison with a reference value. In this case, the processing unit 10 performs face recognition processing using the face recognition model only if the distances D1 and D2 calculated as the degree of fit are smaller than the first and second thresholds of the reference value, respectively, and the face orientation score using the face orientation detection model is higher than a predetermined threshold (facing in a direction close to the front). Furthermore, the process of using a face orientation score obtained using a face orientation detection model as a degree of fit for comparison with a reference value, in addition to the distances D1 and D2 mentioned above, is a specific example of the process described in the claim, which "calculates the degree of fit of the face to be detected as a face authentication target based on the face orientation detected by the face orientation detection means, in addition to the position and size of the head and face."

[0084] In the example shown in Figure 6, the head detection model M11 for detecting a person's head in the frame image and the face detection model M12 for detecting the face region were described as outputting a rectangle representing the area where the head is visible as the detection result. However, the head detection model M11 and the face detection model M12 may each output a bounding box that is not limited to a rectangle, such as a square or an elliptical bounding box, as the detection result.

[0085] In the example shown in Figure 6, a head detection model M11, a face detection model M12, and an age recognition model M13 are used as the learning model group M1 to recognize a person's age. Rectangles R1 and R2 are detected, and the distance between rectangles R1 and R2 is calculated as the degree of fitness. However, if the object to be recognized is different, the method of calculating the degree of fitness will naturally be different, as will the reference value. Therefore, the reference value may be selected by the cloud server 3 when the learning model group M1 is selected and stored in the memory unit 11, and stored together with it.

[0086] For example, when using a person detection model that detects the entire person and a foot detection model that detects the feet, and recognizing the type or color of shoes from the feet, it is preferable that the positional relationship between the rectangle surrounding the area where the detected person is visible and the rectangle surrounding the area where the feet are visible is such that the feet are positioned off-center relative to the area where the entire person is visible, and both feet are detected. In this case, the distance between the vertices of the rectangle should be longer in the vertical direction, but shorter in the approximately horizontal direction. Therefore, the reference value is set to a value different from the first and second thresholds shown in Figure 6. Also, when the detection target is a vehicle, and a vehicle detection model that detects the entire vehicle body and a plate detection model that detects the license plate are used to recognize the vehicle number, it is preferable that the rectangle surrounding the area where the license plate is visible occupies a small area relative to the area of ​​the entire vehicle body, and the reference value should be set appropriately according to such conditions. When the detection target is items on trays such as sorters in a logistics warehouse, and the purpose is to recognize the type of item, it is possible to determine whether the frame image captures a range in which the feature quantities for identifying the item can be appropriately calculated by setting a reference value for the degree of fit.

[0087] [Second Embodiment] In the second embodiment, the group of learning models M1 used in the edge device 1 can be modified as appropriate from the group of models held in the model database 310 accessible by the cloud server 3. Figure 7 is a schematic diagram of the object recognition system 100 of the second embodiment. The hardware configuration of the object recognition system 100 of the second embodiment is the same as that of the object recognition system 100 of the first embodiment, so the same reference numerals are used for common components and detailed explanations are omitted.

[0088] In the object recognition system 100 of the second embodiment, the cloud server 3 holds a group of learning models used by the edge device 1 in the model database 310. As a cloud manager, the cloud server 3 selects a learning model from the model database 310 according to the detection target and recognition target on the edge device 1 and deploys it to the edge device 1. The selection may be made by the client 4 via the cloud server 3, or it may be made by processing based on a predetermined algorithm of the cloud server 3.

[0089] The model database 310 may be built in the storage unit 31 or in an external storage device. A portion of the model database 310 may include a model provision service used on the Web, which is communicated via the network N. The model database 310 holds detection models, such as a person detection model and a vehicle detection model, which are trained to determine whether a specific person or object is present in an image based on the features obtained from the image. The model database 310 holds recognition models that can provide multiple recognition targets. The model database 310 holds attribute-specific models that recognize a person's gender and age as attributes, respectively.

[0090] Figure 8 is a flowchart showing an example of the model setting process procedure in the object recognition system 100 of the second embodiment. When the operator accesses the cloud server 3 using the client 4, the processing unit 30 of the cloud server 3 starts the following process.

[0091] The processing unit 30 identifies the identification data of the edge device 1 that is authorized to be accessed by the account of the operator using client 4, or the identification data or name of the corresponding space (step S301). The processing unit 30 sends a web page containing a list of the identified identification data or name to client 4 (step S302), and accepts the selection of the target edge device 1 (space) from the list on the web page (step S303).

[0092] The processing unit 30 sends a web page containing a screen for accepting the selection of detection targets and recognition targets to the client 4 (step S304), and accepts the selection of detection targets and recognition targets on the web page displayed on the client 4 (step S305). The processing unit 30 selects a detection model and a recognition model from the model database 310 according to the selected detection targets and recognition targets (step S306), and reads the setting of reference values ​​corresponding to the selected detection models and recognition models from the data stored in the storage unit 31 (step S307).

[0093] The processing unit 30 transmits the detection model and recognition model selected in step S306, and the reference value settings read in step S307, to the edge device 1 selected in step S303 (step S308). The processing unit 30 deploys the selected detection model and recognition model, as well as the executable files using them, to the edge device 1 (step S309), and terminates the configuration process.

[0094] The processing procedure shown in Figure 8 can be executed from client 4 at any time. It may be executed during the initial setup of edge device 1, or when the placement of camera 2 is changed in the space where camera 2 is installed.

[0095] Alternatively, using the detection and recognition models deployed to the edge device 1 in step S309, the "process of determining whether the degree of fit of the detection target in each frame image clears the reference value" shown in step S107 of Figure 5 may be performed based on the reference value transmitted in step S308. As a result, only the frame images whose degree of fit clears the reference value may be saved in the storage unit 11 of the edge device 1, or transferred and saved to the cloud server 3. This makes it possible to obtain training images (or images for fine-tuning) for detection and recognition models of the same type as the detection and recognition models deployed to the edge device 1 in step S309.

[0096] [Third Embodiment] In the third embodiment, the learning model group M1 used in the edge device 1 includes a Vision-Language Model (VLM) as a recognition model, which accepts text in addition to image data and can change the processing of image data based on the text. This eliminates the need for the edge device 1 to change the recognition model itself in the event of changes or additions to the recognition targets. Furthermore, even if there are multiple recognition targets, the recognition process can be executed with a single VLM. The learning model group M1 may also include a Multimodal Language Model. In addition to the processing unit 10 and the image recognition program P1 (see Figure 2), the functions of the VLM itself correspond to the "recognition processing modification means" in the claims.

[0097] The hardware configuration of the object recognition system 100 in the third embodiment is the same as that of the object recognition system 100 in the first or second embodiment; therefore, common components are denoted by the same reference numerals and detailed descriptions are omitted. In the third embodiment, the detection model is selected from the model database 310 via the cloud server 3 and deployed to the edge device 1, similar to the second embodiment.

[0098] Figure 9 is a flowchart showing an example of the model setting process in the object recognition system 100 of the third embodiment. When the operator accesses the cloud server 3 using the client 4, the processing unit 30 of the cloud server 3 starts the following process. Of the processing steps shown in Figure 9, steps that are common with the processing steps shown in Figure 8 of the second embodiment are given the same step numbers and detailed explanations are omitted.

[0099] When the processing unit 30 receives the selection of a target edge device 1 (space) from the list on the web page (S303), it sends a web page containing a screen for accepting the selection of a detection target to the client 4 (step S314), and accepts the selection of a detection target on the web page displayed on the client 4 (step S315). The processing unit 30 selects a detection model from the model database 310 according to the selected detection target (step S316), and reads the setting of reference values ​​corresponding to the selected detection model from the data stored in the storage unit 31 (step S317).

[0100] The processing unit 30 receives text to be input to the recognition model, which is a VLM, on the web page displayed on the client 4 (step S318). In step S318, the processing unit 30 receives text such as "Age of the person to be detected" or "How old is the detected person?" in English or any other language.

[0101] The processing unit 30 sends the detection model selected in step S316, the reference value settings read in step S317, and the recognition model text received in step S318 to the edge device 1 (step S319). The processing unit 30 deploys the selected detection model and the executable file using it to the edge device 1 (step S320), and terminates the configuration process.

[0102] The text received by client 4 and sent from cloud server 3 in step S319 is received by edge device 1 and stored in association with the recognition model. The processing unit 10 of edge device 1 inputs the acquired frame image into a detection model of two or more parts, and if the degree of fit calculated based on the two detection results clears the conditions using a reference value, it inputs the frame image into the recognition model which is VLM. The processing unit 10 of edge device 1 inputs the text specifying the recognition target received from client 4 via cloud server 3 into the recognition model which is VLM, and obtains the recognition result output from the recognition model. If there are multiple recognition targets, for example, age and gender, the processing unit 10 inputs the text "Output the age of the detected person" and the text "Output the gender of the detected person," and obtains the recognition result including age, gender, and confidence score (of the recognition result).

[0103] As shown in Figure 9, the recognition model used in edge device 1 can change the recognition target by modifying the text at any time in response to instructions from the operator received by client 4.

[0104] Figure 10 is an explanatory diagram of the processing performed by the edge device 1 of the third embodiment. Figure 10 shows an example in which the edge device 1 uses a head detection model M11 and a face detection model M12 for the purpose of recognizing a person's age and gender, similar to the processing content shown in Figure 6. The edge device 1 of the third embodiment uses a VLM model M14 as the recognition model. If the degree of fit (distance D1, D2) meets the conditions using the reference value, the processing unit 10 inputs the first or second part of the frame image to the model M14, and inputs text instructing the output of age and text instructing the output of gender to the model M14. The processing unit 10 acquires the recognition result (age and gender, and confidence score) output from the model M14 and stores it together with the degree of fit. The processing unit 10 may also send the recognition result to the cloud server 3 in association with the identification data of the frame image.

[0105] In the third embodiment, since the recognized content can be changed by text, there is no need to replace the recognition model in accordance with the change in recognized content.

[0106] [Fourth Embodiment] In the fourth embodiment, the group of learning models M1 used in the edge device 1 is divided into a backbone learning model that extracts features from the input image data and a task head learning model that performs recognition processing based on the extracted features, for both the detection model and the recognition model. In the fourth embodiment as well, the group of learning models M1 used in the edge device 1 can be modified as appropriate from the group of models held in the model database 310 accessible by the cloud server 3.

[0107] The hardware configuration of the object recognition system 100 in the fourth embodiment is the same as that of the object recognition system 100 in the first embodiment; therefore, common components are denoted by the same reference numerals and detailed descriptions are omitted. In the fourth embodiment, the backbone portion of both the (trained) detection model and the recognition model is not replaced, and the task head portion of the detection model and the task head portion of the recognition model are selected and changed from the model database 310 via the cloud server 3.

[0108] Figure 11 is a flowchart showing an example of the model setting process in the object recognition system 100 of the fourth embodiment. When the operator accesses the cloud server 3 using the client 4, the processing unit 30 of the cloud server 3 starts the following process. Of the processing steps shown in Figure 11, steps that are common with the processing steps shown in Figure 8 of the second embodiment are given the same step numbers and detailed explanations are omitted.

[0109] In the fourth embodiment, when the processing unit 30 receives the selection of a detection target and a recognition target (S305), it selects a learning model for the corresponding task head portion according to the selected detection target and recognition target, respectively (step S326). The processing unit 30 reads the setting of reference values ​​corresponding to the selected task head portion learning model from the data stored in the storage unit 31 (step S327).

[0110] The processing unit 30 sends the learning model of the task head portion selected in step S326 and the reference value settings read in step S327 to the selected edge device 1 (step S328). The processing unit 30 deploys the learning model of the selected task head portion and the executable file using them to the edge device 1 (step S329), and terminates the configuration process.

[0111] Figure 12 is an explanatory diagram of the processing performed by the edge device 1 in the fourth embodiment. Similar to the processing shown in Figure 6, Figure 12 shows an example in which the edge device 1 uses a head detection model M11, a face detection model M12, and an age recognition model M13 for the purpose of recognizing a person's age and gender. In the fourth embodiment, the head detection model M11 and the face detection model M12 are task head models. The head detection model M11 and the face detection model M12 are configured to perform head detection and face detection, respectively, using feature data obtained from the backbone model M11B. The age recognition model M13 is also a task head model and outputs recognition results using features obtained from the backbone model M13B.

[0112] In the fourth embodiment, the processing unit 30 inputs the frame image to model M11B, outputs a first detection result from the head detection model M11 using the features calculated by model M11B, and outputs a second detection result from the face detection model M12. Thereafter, the calculation of the degree of fit using the first and second detection results is the same as in the first embodiment.

[0113] In the fourth embodiment, the operator can refer to the recognition results from the edge device 1 via the cloud server 3 using the client 4, and if they wish to change the detection content and recognition content, they can swap the task head portion. In this case, as shown at the top of Figure 12, the detection model can be changed to a person detection model M15 in the task head portion that detects the entire person, and a face detection model M16 that detects the face portion from the entire person, and the recognition model can be changed to a gender recognition model M17.

[0114] Thus, in the fourth embodiment, the detection target and recognition target can be changed by replacing only the task head portion, eliminating the need to replace the entire recognition model in response to changes in the recognition content. Depending on what the detection target is and what the content (target) to be recognized for the detection target is, a recognition system that can accommodate a variety of conditions can be realized by replacing only the necessary parts as much as possible, without having to replace the entire system.

[0115] The embodiments disclosed above are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the claims, and all modifications within the meaning and scope equivalent to the claims are included. [Explanation of Symbols]

[0116] 100 Object Recognition Systems 1. Edge devices 10 Processing Unit 11 Storage section P1 Image Recognition Program M1 Learning Model Group M11 Head Detection Model M12 Face Detection Model M13 Age Recognition Model M14 Model 2 cameras 3. Cloud Server 30 Processing Unit 31 Storage section 300,310 databases 4 Clients 40 Processing Unit 43 Display section

Claims

1. An object detection means for detecting two or more parts of an object to be detected that are captured in a frame image input from a camera, A fitness calculation means that calculates the fitness of the object to be detected as a recognition target based on the position and size of the two or more parts, A comparison means for comparing the degree of suitability as the object of recognition with a predetermined standard value, An object recognition system comprising object recognition means that recognizes only the objects to be detected that meet the criteria value as a result of the above comparison.

2. The object detection means detects two or more parts of the object to be detected by determining the bounding boxes of two or more parts of the object to be detected. The object recognition system according to claim 1, characterized in that the fittingness calculation means calculates the fittingness of the object to be detected as a recognition target based on the positions and sizes of the bounding boxes of the two or more parts obtained by the object detection means.

3. The object detection means detects two rectangular portions of the object to be detected, The two rectangular portions are such that one rectangular portion encloses the other rectangular portion. The object recognition system according to claim 1, characterized in that the fittingness calculation means determines the distance between a predetermined vertex in one rectangle and a vertex in the other rectangle corresponding to the predetermined vertex, and calculates the fittingness of the object to be detected as a recognition target based on this distance.

4. The object detection means and the object recognition means are a trained object detection model and a trained object recognition model, The object recognition system according to claim 1, further comprising a modification means for modifying the trained object detection model, the trained object recognition model, and the reference value in response to instructions from an operator via the cloud.

5. The object recognition means is a vision language model, A text modification means that modifies the input text to the vision language model in response to instructions from an operator via the cloud, The object recognition system according to claim 1, further comprising recognition processing modification means for changing the content of object recognition processing by the vision language model according to the input text.

6. The object recognition system according to claim 1, characterized in that the degree of suitability of the object to be detected as a recognition target is used as the reliability of the recognition result of the object to be detected.

7. The object recognition system according to claim 5, characterized in that the confidence score of the object recognition result by the vision language model and the degree of fit of the object to be detected as a recognition target are used as the confidence score of the recognition result of the object to be detected.

8. The object recognition system according to claim 4, characterized in that the modification means modifies the trained object detection model and the trained object recognition model by replacing only the task head portion without replacing the backbone portion that extracts feature quantities from frame images for both the trained object detection model and the trained object recognition model.

9. A head detection means for detecting a person's head captured in a frame image input from a camera, A face detection means for detecting the face of a person captured in the frame image, A face orientation detection means for detecting the face orientation of the face detected by the face detection means, A suitability calculation means calculates the suitability of the face to be detected as a face recognition target based on the position and size of the head and face, as well as the face orientation detected by the face orientation detection means. A comparison means for comparing the degree of suitability as a facial recognition target with a predetermined standard value, The system includes a face recognition means that performs face recognition processing on the face detected by the face detection means, The facial recognition means is an object recognition system that performs facial recognition processing only on faces of targets that meet the criteria value as a result of comparison by the comparison means.

10. Computers, An object detection means for detecting two or more parts of an object to be detected that are captured in a frame image input from a camera, A fitness calculation means that calculates the fitness of the object to be detected as a recognition target based on the position and size of the two or more parts, A comparison means for comparing the degree of suitability as the object of recognition with a predetermined standard value, An object recognition program that functions as an object recognition means for recognizing only the objects to be detected that meet the criteria based on the results of the above comparison.

11. Computers, A head detection means for detecting a person's head captured in a frame image input from a camera, A face detection means for detecting the face of a person captured in the frame image, A face orientation detection means for detecting the face orientation of the face detected by the face detection means, A suitability calculation means calculates the suitability of the face to be detected as a face recognition target based on the position and size of the head and face, as well as the face orientation detected by the face orientation detection means. A comparison means for comparing the degree of suitability as a facial recognition target with a predetermined standard value, An object recognition program for functioning as a face recognition means that performs face recognition processing on a face detected by the face detection means, The facial recognition means is an object recognition program that performs facial recognition processing only on the faces of the target to be detected that meet the criteria value as a result of the comparison by the comparison means.

Images