Service providing method and apparatus

Abstract

This specification provides a service providing method and apparatus. The method includes: obtaining a first image of a target area; performing human body detection on the first image to obtain at least one human body detection result, where the human body detection result includes a human body detection frame; classifying the image area corresponding to the human body detection frame in multiple dimensions, and obtaining a target classification result of the object corresponding to the human body detection frame according to the classification results of each dimension; outputting first information according to the target classification result of the object; where the first information indicates to provide a specific service, ensuring the timeliness of service provision.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and particularly to a service providing method and apparatus. Background Art

[0002] In daily work and life, people often need to go to business halls to handle corresponding services. To better serve people, the staff in business halls often provide caring services to special groups (such as the elderly, pregnant women, and disabled people), such as giving priority to queuing numbers.

[0003] Currently, when determining whether caring services need to be provided, the staff need to first determine whether there are special groups of people who need caring services in the business hall. If there are, the staff provide caring services to such special groups.

[0004] However, due to some reasons (such as a large number of people in the business hall), the staff cannot timely discover special groups of people who need caring services, and thus cannot provide services in a timely manner. Summary of the Invention

[0005] To overcome the problems existing in the related art, this specification provides a service providing method and apparatus.

[0006] According to a first aspect of the embodiments of this specification, a service providing method is provided. The method includes:

[0007] Obtain a first image of a target area;

[0008] Perform human detection on the first image to obtain at least one human detection result, where the human detection result includes a human detection frame;

[0009] Classify the image area corresponding to the human detection frame in multiple dimensions, and obtain a target classification result of the object corresponding to the human detection frame according to the classification results of each dimension;

[0010] Output first information according to the target classification result of the object; where the first information indicates to provide a specific service.

[0011] Optionally, the target classification result indicates the type of the object;

[0012] The outputting first information according to the target classification result of the object includes:

[0013] When the target classification result of the object indicates that the type of the object is a first preset type, determine a companion detection result corresponding to the object according to a first object other than the object in the first image; where the first preset type represents the type of the object for which a specific service is provided;

[0014] Output the first information according to the detection result of the accompanying person corresponding to the object.

[0015] Optionally, determining the detection result of the accompanying person corresponding to the object according to the first object other than the object in the first image includes:

[0016] Based on the human detection frame corresponding to the object and the human detection frames corresponding to each first object, determine the distances between the object and each first object;

[0017] When the distances between the object and each first object are all greater than the preset accompanying distance, determine that the detection result of the accompanying person corresponding to the object indicates that there is no accompanying person for the object;

[0018] When there is a distance between an object and the first object that is less than or equal to the preset accompanying distance, determine that the detection result of the accompanying person corresponding to the object indicates that there is an accompanying person for the object.

[0019] Optionally, outputting the first information according to the detection result of the accompanying person corresponding to the object includes:

[0020] When the detection result of the accompanying person corresponding to the object indicates that there is no accompanying person for the object, determine the target terminal corresponding to the object;

[0021] Output the first information to the target terminal; wherein, the first information includes a prompt message and / or control information; the prompt message is used to be displayed on the target terminal to prompt to provide a specific service; the control information is used to control the target terminal to perform a specific service reminder according to a preset reminder method.

[0022] Optionally, determining the target terminal corresponding to the object includes:

[0023] Obtain the distances between the object and each terminal, and use the terminal with the shortest distance as the target terminal corresponding to the object.

[0024] Optionally, the method further includes:

[0025] Perform face detection on the image area corresponding to the human detection frame to obtain the face detection frame corresponding to the object, and based on the face recognition algorithm, match the face detection frame corresponding to the object with the face information in the preset information library;

[0026] When the face detection box corresponding to the object successfully matches the face information, determine the prompt information according to the record information corresponding to the face information in the preset information library; wherein, the prompt information includes an object identifier and / or historical service handling information; the historical service handling information indicates the services that the object has handled.

[0027] Optionally, the matching of the face detection box corresponding to the object with the face information in the preset information library includes:

[0028] Perform face alignment processing on the face detection box;

[0029] Determine the image quality information corresponding to the aligned face detection box;

[0030] When it is determined that the image quality information meets the preset quality conditions, match the aligned face detection box with the face information in the preset information library.

[0031] Optionally, the first image is a frame in an image sequence; the image sequence includes a preset number of continuously captured images;

[0032] The method further includes:

[0033] For any dimension, determine whether there is a second object identical to the object in a second image; wherein, the second image is any frame other than the first image in the image sequence; the first type indicates the type in the dimension;

[0034] When there is a second object identical to the object, obtain the classification result of the second object in the dimension;

[0035] Obtain the first number of classification results indicating that the first type of the second object is a second preset type;

[0036] When the first number is less than the preset number, adjust the classification result of the object in the dimension.

[0037] Optionally, the determining whether there is a second object identical to the object in the second image includes:

[0038] Based on the image region corresponding to the human detection box corresponding to the object, determine the feature data corresponding to the object, and obtain the feature data corresponding to each second object in the second image;

[0039] For each second object, determine the Euclidean distance between the object and the second object according to the feature data corresponding to the object and the feature data corresponding to the second object;

[0040] When the Euclidean distance is less than a preset distance threshold, determine that the second object is the same second object as the object.

[0041] Optionally, when the Euclidean distance is less than a preset distance threshold, determining that the second object is the same second object as the object includes:

[0042] When the Euclidean distance is less than a preset distance threshold, crop the image region corresponding to the human detection box of the object to obtain a plurality of first image patches, and crop the image region corresponding to the human detection box of the second object to obtain a plurality of second image patches;

[0043] For each first image patch, respectively determine the similarity between the first image patch and each second image patch, and determine the number of second image patches with a similarity greater than a first preset similarity;

[0044] Use the number of second image patches with a similarity greater than the first preset similarity as the second number corresponding to the first image patch;

[0045] Based on the second numbers corresponding to each first image patch, determine that the second object is the same second object as the object.

[0046] Optionally, based on the second numbers corresponding to each first image patch, determining that the second object is the same second object as the object includes:

[0047] Calculate the sum of the second numbers corresponding to all first image patches to obtain a similarity number;

[0048] When the similarity number is greater than a preset similarity number, determine the distance difference between the center point of the human detection box of the object and the center point of the human detection box of the second object; wherein, the distance difference includes a width difference and / or a height difference;

[0049] Based on the distance difference, determine that the second object is the same second object as the object.

[0050] Optionally, the dimension includes one or more of a disabled dimension, a pregnant woman dimension, and an elderly dimension;

[0051] The classifying the image region corresponding to the human detection box into multiple dimensions includes:

[0052] Using a first preset classification model, perform disabled person classification based on the image region corresponding to the human detection box to obtain a classification result corresponding to the disabled person dimension; wherein, the first preset recognition model is used to recognize whether the object corresponding to the human detection box belongs to a disabled person;

[0053] And / or,

[0054] Using a second preset classification model, perform pregnant woman classification on the image region corresponding to the human detection frame to obtain a classification result corresponding to the pregnant woman dimension; wherein, the second preset classification model is used to identify whether the object corresponding to the human detection frame belongs to a pregnant woman;

[0055] And / or,

[0056] Perform face detection on the image region corresponding to the human detection frame to obtain a face detection frame;

[0057] Using a third preset classification model, perform elderly person classification on the image region corresponding to the face detection frame to obtain a classification result corresponding to the elderly person dimension; wherein, the third preset classification model is used to identify whether the object corresponding to the face detection frame belongs to an elderly person.

[0058] According to the second aspect of the embodiments of the present specification, a service providing device is provided, including:

[0059] An image acquisition module, configured to acquire a first image of a target area;

[0060] A detection module, configured to perform human detection on the first image to obtain at least one human detection result, and the human detection result includes a human detection frame;

[0061] A classification module, configured to perform classification on the image region corresponding to the human detection frame in multiple dimensions, and obtain a target classification result of the object corresponding to the human detection frame according to the classification results of each dimension;

[0062] A processing module, configured to output first information according to the target classification result of the object; wherein, the first information indicates to provide a specific service.

[0063] Optionally, the target classification result indicates the type of the object;

[0064] The processing module is specifically configured to:

[0065] In the case where the target classification result of the object indicates that the type of the object is a first preset type, determine a detection result of an accompanying person corresponding to the object according to a first object other than the object in the first image; wherein, the first preset type represents the type of the object for which a specific service is provided;

[0066] Output first information according to the detection result of the accompanying person corresponding to the object.

[0067] Optionally, the processing module is further configured to:

[0068] Determine the distances between the object and each first object based on the human detection frame corresponding to the object and the human detection frames corresponding to each first object;

[0069] When the distances between the object and each first object are all greater than a preset accompanying distance, determine that the accompanying person detection result corresponding to the object indicates that there is no accompanying person for the object;

[0070] When there is a distance between an object and the first object that is less than or equal to the preset accompanying distance, determine that the accompanying person detection result corresponding to the object indicates that the object has an accompanying person.

[0071] Optionally, the processing module is further configured to:

[0072] When the accompanying person detection result corresponding to the object indicates that there is no accompanying person for the object, determine the target terminal corresponding to the object;

[0073] Output first information to the target terminal; wherein, the first information includes prompt information and / or control information; the prompt information is used to be displayed on the target terminal to prompt to provide a specific service; the control information is used to control the target terminal to perform a specific service reminder according to a preset reminder method.

[0074] Optionally, the processing module is further configured to:

[0075] Obtain the distances between the object and each terminal, and use the terminal with the shortest distance as the target terminal corresponding to the object.

[0076] Optionally, the detection module is further configured to:

[0077] Perform face detection on the image region corresponding to the human detection frame to obtain the face detection frame corresponding to the object, and based on the face recognition algorithm, match the face detection frame corresponding to the object with the face information in the preset information database;

[0078] When the face detection frame corresponding to the object matches the face information successfully, determine the prompt information according to the record information corresponding to the face information in the preset information database; wherein, the prompt information includes object identification and / or historical business handling information; the historical business handling information indicates the business that the object has handled.

[0079] Optionally, the detection module is further configured to:

[0080] Perform face alignment processing on the face detection frame;

[0081] Determine the image quality information corresponding to the aligned face detection frame;

[0082] When it is determined that the image quality information meets the preset quality condition, the aligned face detection frame is matched with the face information in the preset information library.

[0083] Optionally, the first image is a frame in the image sequence; the image sequence includes a preset number of continuously captured images;

[0084] The classification module is further configured to:

[0085] For any dimension, determine whether there is a second object in the second image that is the same as the object; wherein, the second image is any frame in the image sequence other than the first image; the first type indicates the type in the dimension;

[0086] When there is a second object that is the same as the object, obtain the classification result of the second object in the dimension;

[0087] Obtain the first number of classification results indicating that the first type of the second object is the second preset type;

[0088] When the first number is less than the preset number, adjust the classification result of the object in the dimension.

[0089] Optionally, the classification module is further configured to:

[0090] Based on the image region corresponding to the human detection frame corresponding to the object, determine the feature data corresponding to the object, and obtain the feature data corresponding to each second object in the second image;

[0091] For each second object, determine the Euclidean distance between the object and the second object according to the feature data corresponding to the object and the feature data corresponding to the second object;

[0092] When the Euclidean distance is less than the preset distance threshold, determine that the second object is the second object that is the same as the object.

[0093] Optionally, the classification module is further configured to:

[0094] When the Euclidean distance is less than the preset distance threshold, crop the image region corresponding to the human detection frame corresponding to the object to obtain a plurality of first image blocks, and crop the image region corresponding to the human detection frame corresponding to the second object to obtain a plurality of second image blocks;

[0095] For each first image block, determine the similarity between the first image block and each second image block, and determine the number of second image blocks with a similarity greater than the first preset similarity;

[0096] Take the number of second image patches whose similarity is greater than the first preset similarity as the second number corresponding to the first image patch;

[0097] Determine that the second object is the same second object as the object according to the second numbers corresponding to each first image patch.

[0098] Optionally, the classification module is further configured to:

[0099] Calculate the sum of the second numbers corresponding to all first image patches to obtain a similarity number;

[0100] When the similarity number is greater than a preset similarity number, determine the distance difference between the center point of the human detection frame of the object and the center point of the human detection frame of the second object; wherein, the distance difference includes a width difference and / or a height difference;

[0101] Determine that the second object is the same second object as the object according to the distance difference.

[0102] Optionally, the dimension includes one or more of a disabled person dimension, a pregnant woman dimension, and an elderly person dimension;

[0103] The classification module is specifically configured to:

[0104] Use a first preset classification model to perform disabled person classification based on the image region corresponding to the human detection frame to obtain a classification result corresponding to the disabled person dimension; wherein, the first preset recognition model is used to identify whether the object corresponding to the human detection frame belongs to a disabled person;

[0105] And / or,

[0106] Use a second preset classification model to perform pregnant woman classification based on the image region corresponding to the human detection frame to obtain a classification result corresponding to the pregnant woman dimension; wherein, the second preset recognition model is used to identify whether the object corresponding to the human detection frame belongs to a pregnant woman;

[0107] And / or,

[0108] Perform face detection on the image region corresponding to the human detection frame to obtain a face detection frame;

[0109] Use a third preset classification model to perform elderly person classification on the image region corresponding to the face detection frame to obtain a classification result corresponding to the elderly person dimension; wherein, the third preset recognition model is used to identify whether the object corresponding to the face detection frame belongs to an elderly person.

[0110] According to the third aspect of the embodiments of the present specification, there is provided an electronic device, including:

[0111] A processor;

[0112] A memory for storing processor-executable instructions;

[0113] Wherein, the processor is configured to:

[0114] Obtain a first image of a target area;

[0115] Perform human body detection on the first image to obtain at least one human body detection result, where the human body detection result includes a human body detection frame;

[0116] Classify the image area corresponding to the human body detection frame in multiple dimensions, and obtain a target classification result of the object corresponding to the human body detection frame according to the classification results of each dimension;

[0117] Output first information according to the target classification result of the object; wherein, the first information indicates to provide a specific service.

[0118] According to the fourth aspect of the embodiments of the present specification, there is provided a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the service providing method described in the first aspect above and various possible designs of the first aspect is implemented.

[0119] According to the fifth aspect of the embodiments of the present specification, there is provided a computer program product, including a computer program, which when executed by a processor, implements the service providing method described in the first aspect above and various possible designs of the first aspect.

[0120] The technical solutions provided by the embodiments of the present specification may include the following beneficial effects: By performing human body detection on the first image of the target area, a human body detection result is obtained, that is, a human body detection frame corresponding to the object in the first image is obtained. Classify the image area corresponding to the human body detection frame corresponding to the object in multiple dimensions to obtain the classification results of the object in each dimension, and use the classification results of the object in each dimension to determine the target classification result of the object, that is, to determine whether the object is a special person who needs a specific service (for example, a caring service), so as to quickly and accurately determine the special person. Based on the target classification result of the object, that is, when the object is a special person who needs a specific service, output the first information, so that relevant staff can provide the specific service in time, ensure the timeliness of service provision, and provide the experience of specific personnel.

[0121] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit this specification. Brief Description of the Drawings

[0122] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with this specification, and are used together with the specification to explain the principles of this specification.

[0123] Figure 1 It is a flowchart of a service providing method shown according to an exemplary embodiment of this specification.

[0124] Figure 2 It is a schematic diagram of a human detection frame shown according to an exemplary embodiment of this specification.

[0125] Figure 3 It is a flowchart of another service providing method shown according to an exemplary embodiment of this specification.

[0126] Figure 4 It is a schematic diagram of an elderly person shown according to an exemplary embodiment of this specification.

[0127] Figure 5 It is a hardware structure diagram of an electronic device where the service providing device in the embodiment of this specification is located.

[0128] Figure 6 It is a block diagram of a service providing device shown according to an exemplary embodiment of this specification. Detailed implementation manners

[0129] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0130] The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit this specification. The singular forms "a", "the", and "said" used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0131] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to a determination".

[0132] Next, the embodiments of this specification will be described in detail.

[0133] As Figure 1 shown, Figure 1 is a flowchart of a service providing method shown in this specification according to an exemplary embodiment, including the following steps:

[0134] Step 101, obtain a first image of a target area.

[0135] In this embodiment, when people go to places such as business halls to handle corresponding businesses, in order to better serve people, the staff in the place provide specific services (for example, caring services) to special personnel among the people handling the business. Therefore, an image of the target area corresponding to the place (i.e., the first image) can be collected for determining whether there are special personnel in the target area by using the first image, so that when it is determined that there are special personnel, services can be provided to the special personnel in a timely manner.

[0136] Among them, the target area corresponding to the place may be the entrance area, indoor area, etc. of the place. The imaging device takes pictures of the target area corresponding to the place to obtain a corresponding video, which includes multiple frames of images, and the first image is one frame of the multiple frames of images.

[0137] Step 102, perform human body detection on the first image to obtain at least one human body detection result, and the human body detection result includes a human body detection box.

[0138] In this embodiment, based on a human body detection model, human body detection is performed on the first image to determine each person included in the first image, that is, the human body area of the object, so as to obtain a human body detection box corresponding to each object, and the human body area indicates the entire body tissue of the object (such as Figure 2 the human body detection box of object 1 and the human body detection box of object 2 shown in the first image).

[0139] Step 103, classify the image area corresponding to the human body detection box in multiple dimensions, and obtain a target classification result of the object corresponding to the human body detection box according to the classification results of each dimension.

[0140] In this embodiment, for each object in the first image, based on the image region corresponding to the human detection box corresponding to the object, the object is classified in multiple dimensions to determine the classification results of the object in each dimension. The classification results of the object in each dimension are synthesized to obtain the target classification result of the object, so as to accurately determine the object type.

[0141] Optionally, the classification result of the object in a dimension indicates the type of the object in that dimension (i.e., the first type). The dimensions include one or more of the disabled dimension, the pregnant woman dimension, and the elderly dimension. For example, when the dimension includes the disabled dimension, the classification result of the object in the disabled dimension indicates the type of the object in the disabled dimension, that is, it indicates whether the object is disabled; for another example, when the dimension includes the pregnant woman dimension, the classification result of the object in the pregnant woman dimension indicates the type of the object in the pregnant woman dimension, that is, it indicates whether the object is pregnant; when the dimension includes the elderly dimension, the classification result of the object in the elderly dimension indicates the type of the object in the elderly dimension, that is, it indicates whether the object is elderly. Of course, the dimensions can also be set according to actual needs. For example, the dimension is set to include the child dimension.

[0142] Optionally, the target classification result indicates the type of the object, that is, the comprehensive type of the object. For example, when the dimensions include the disabled dimension, the pregnant woman dimension, and the elderly dimension, the classification result of the object in the disabled dimension indicates that the object is disabled, the classification result of the object in the pregnant woman dimension indicates that the object is not pregnant, and the classification result of the object in the elderly dimension indicates that the object is elderly. Then the target classification result indicates that the object is of the disabled type and the elderly type, that is, it indicates that the object is both disabled and elderly.

[0143] Step 104: Output the first information according to the target classification result of the object. The first information indicates to provide a specific service.

[0144] In this embodiment, after obtaining the target classification result of the object, it can be determined whether the object is a special person who needs a specific service. Thus, when the object is the special person, the first information is output, so that the staff can provide services to the object in a timely manner.

[0145] As can be seen from the above description, through human body detection on the first image of the target area, a human body detection result is obtained, that is, a human body detection box corresponding to the object in the first image is obtained. The image area corresponding to the human body detection box corresponding to the object is classified in multiple dimensions to obtain the classification results of the object in each dimension, and the target classification result of the object is determined by using the classification results of the object in each dimension, that is, it is determined whether the object is a special person who needs a specific service (for example, a caring service), so as to realize the rapid and accurate determination of special persons. Based on the target classification result of the object, that is, when the object is a special person who needs a specific service, the first information is output, so that relevant staff can provide specific services in time, ensure the timeliness of service provision, and provide the experience of specific persons.

[0146] As Figure 3 shown, Figure 3 is a flowchart of another service provision method shown according to an exemplary embodiment. On the basis of the foregoing embodiment, this embodiment describes the process of determining whether to provide a specific service. The following will describe this process in detail with a specific example. As Figure 3 shown, the method includes the following steps: includes the following steps:

[0147] Step 301, obtain the first image of the target area.

[0148] Step 302, perform human body detection on the first image to obtain at least one human body detection result, and the human body detection result includes a human body detection box.

[0149] Step 303, classify the image area corresponding to the human body detection box in multiple dimensions, and obtain the target classification result of the object corresponding to the human body detection box according to the classification results of each dimension.

[0150] Optionally, when the dimension includes the disabled dimension, when determining the classification result of the object in the disabled dimension, that is, when determining whether the object belongs to the disabled, use the first preset classification model to perform disabled classification based on the image area corresponding to the human body detection box to obtain the classification result corresponding to the disabled dimension. Among them, the first preset classification model is used to identify whether the object corresponding to the human body detection box belongs to the disabled.

[0151] Among them, the first preset classification model refers to a disabled detection model, which performs disabled detection on the image area corresponding to the human body detection box of the object to obtain the classification result of the object in the disabled dimension, that is, to determine whether the object belongs to the disabled. Specifically, when using the disabled detection model to detect the image area corresponding to the object, first obtain the label result, and determine the classification result of the object in the disabled dimension by synthesizing the label result.

[0152] Optionally, the label result includes identification 1 (for example, 0), identification 2 (for example, 1), identification 3 (for example, 2), identification 3 (for example, 2), identification 4 (for example, 3). Among them, identification 1 indicates a disabled person sitting in a wheelchair, identification 2 indicates a disabled person using crutches or a blind stick, identification 3 indicates a disabled person pushing a wheelchair, and identification 4 indicates a normal person. When the label result is identification 1, identification 2, or identification 3, it is determined that the classification result of the object in the disabled person dimension indicates that the first type of the object is a disabled person, that is, it indicates that the object belongs to a disabled person. When the label result is identification 4, it is determined that the classification result of the object in the disabled person dimension indicates that the first type of the object is not a disabled person, that is, it indicates that the object belongs to a normal person.

[0153] Optionally, when the dimension includes a pregnant woman dimension, when determining the classification result of the object in the pregnant woman dimension, the second preset classification model is used to classify the pregnant woman based on the image area corresponding to the human body detection frame to obtain the classification result corresponding to the pregnant woman dimension. The second preset classification model is used to identify whether the object corresponding to the human body detection frame is a pregnant woman.

[0154] Among them, the second preset classification model indicates the pregnant woman detection model, which performs pregnant woman detection on the image area corresponding to the human body detection frame corresponding to the object to obtain the classification result of the object in the pregnant woman dimension, that is, to determine whether the object is a pregnant woman.

[0155] Optionally, the pregnant woman model can be a Resnet18 model. Specifically, the improved Resnet18 model is used for classification, the residual block part of the Resnet18 model is improved, a 1*1 convolution layer is added, and the corresponding weight is 0.5, and the residual block of the x input is added, and the corresponding weight value is 0.5. The improved Resnet18 model can effectively improve the classification efficiency and accuracy of pregnant women.

[0156] Optionally, when the dimension includes an elderly dimension, when determining the classification result of the object in the elderly dimension, face detection is performed on the image area corresponding to the human body detection frame to obtain the face detection frame. The image area corresponding to the face detection frame is classified as elderly using a third preset classification model to obtain a classification result corresponding to the elderly dimension. The third preset classification model is used to identify whether the object corresponding to the face detection frame belongs to an elderly person.

[0157] Specifically, since the faces of elderly people have more wrinkles and their muscles are more relaxed, it is possible to determine whether the object is an elderly person based on the face of the object.

[0158] Optionally, the third preset classification model indicates a face age recognition model. By processing the image region corresponding to the face detection box of the object through this model, the age of the object is obtained. When the age is greater than or equal to a preset age (for example, 60 years old), it is determined that the object belongs to the elderly, that is, it is determined that the classification result of the object in the elderly dimension indicates that the first type of the object in the elderly dimension is the elderly type.

[0159] After determining that the object does not belong to the elderly based on the face, in order to avoid inaccurate classification results of the object in the elderly dimension based on the face due to reasons such as the object turning its head or being far away from the camera device, considering that the bending degree of the back of the elderly is relatively large (as Figure 4 shown), therefore, it can be determined whether the object belongs to the elderly based on the human detection box corresponding to the object, that is, using the fourth preset classification model, the elderly classification is performed based on the image region corresponding to the human detection box to obtain the classification result corresponding to the elderly dimension.

[0160] Among them, the fourth preset classification model indicates a human elderly classification model.

[0161] In this embodiment, in order to improve the accuracy of the classification result, multiple images are used to comprehensively judge the classification results of the object in each dimension. Then, an image sequence is obtained from the video captured by the camera device. The image sequence includes a preset number of continuously captured images. The first image is a frame in the image sequence, that is, a certain number of images captured before and after the first image are obtained.

[0162] After determining the image sequence, for any dimension, it is judged whether there is a second object identical to the object in the second image. Among them, the second image is any frame in the image sequence other than the first image. The first type indicates the type in this dimension. In the case where there is a second object identical to the object, the classification result of the second object in this dimension is obtained. The first number of classification results indicating that the first type of the second object is the second preset type is obtained. When the first number is less than the preset number, the classification result of the object in this dimension is adjusted.

[0163] Among them, the second preset type indicates the type of the object that requires specific services corresponding to the dimension. For example, when the dimension is the disabled dimension, the second preset type is the disabled type; for another example, when the dimension is the pregnant woman dimension, the second preset type is the pregnant woman type; for still another example, when the dimension is the elderly dimension, the second preset type is the elderly type.

[0164] Specifically, for the sake of convenience in description, the object in the second image is regarded as the second object. For each object in the first image (i.e., the object corresponding to each human detection box), it is determined whether there is a second object identical to the object in the second image. If there is, the classification result of the second object identical to the object in a certain dimension is obtained, so that the classification results of the second object in each dimension can be obtained.

[0165] For each dimension, the number of classification results indicating that the first type of the second object in this dimension is the second preset type corresponding to this dimension is counted, and this number is taken as the first number. When the first number is less than the preset number, it indicates that the object does not belong to the specific person who needs specific services in this dimension, that is, through the first image, the classification result of the object in this dimension is incorrect, so the classification result of the object in this dimension is adjusted, and the adjusted classification result of the object in this dimension indicates that the first type of the object is not the second preset type corresponding to this dimension. For example, the dimension includes the disabled dimension. The image sequence includes Image 1, Image 2, and Image 3, the first image is Image 3, and Image 3 includes Object 1. It is determined that the classification result of Object 1 in the disabled dimension indicates that Object 1 is disabled. Both Image 1 and Image 2 include Object 1, and it is determined through Image 1 that Object 1 is not disabled, and it is determined through Image 2 that Object 1 is not disabled, then the first number is counted as 0, which is less than the preset number (i.e., 2), and based on Image 3, the classification result of Object 1 in the disabled dimension is incorrect, so the classification result is adjusted to indicate that Object 1 is not disabled.

[0166] Among them, the process of determining the classification result of the second object in a certain dimension is similar to the process of determining the classification result of the object in the first image in a certain dimension, both of which are classified through relevant models, and the process of determining the second object in the second image and the process type of determining the object in the first image are both determined by determining the human detection box. Here, it will not be elaborated.

[0167] Of course, on the basis of obtaining the first number, the percentage can also be determined by using the first number. For example, the dimension is the disabled. When more than 80% of the images in the image sequence determine that the object is disabled, it is determined that the object is disabled. As long as the voting principle is used, that is, the actual classification result corresponding to the object is determined based on the classification results of the objects corresponding to multiple images, it is within the protection scope of this application. Here, it will not be elaborated one by one.

[0168] Optionally, for each object in the first image, the object is tracked, that is, it is determined whether there is a second object identical to the object in the second image. The specific process of this determination is as follows:

[0169] Based on the image region corresponding to the human detection box corresponding to the object, determine the feature data corresponding to the object, and obtain the feature data corresponding to each second object in the second image. For each second object, determine the Euclidean distance between the object and the second object according to the feature data corresponding to the object and the feature data corresponding to the second object. When the Euclidean distance is less than the preset distance threshold, determine that the second object is the same second object as the object, that is, the two belong to the same target.

[0170] Among them, the feature data indicates the number of face features, service features, body features, etc. The determination process of the feature data corresponding to the second object is similar to the determination process of the feature data corresponding to the object in the first image.

[0171] Optionally, when determining the feature data of the object, the first image and the second image can be filtered by a correlation filter.

[0172] Optionally, in order to improve the accuracy of object tracking determination, that is, the accuracy of determining the same target, when the Euclidean distance between the object and the second object is less than the preset distance threshold, continue to determine whether the object and the second object belong to the same target, that is, when the Euclidean distance is less than the preset distance threshold, crop the image region corresponding to the human detection box corresponding to the object to obtain a plurality of first image patches, and crop the image region corresponding to the human detection box corresponding to the second object to obtain a plurality of second image patches. For each first image patch, determine the similarity between the first image patch and each second image patch respectively, and determine the number of second image patches with a similarity greater than the first preset similarity. Use the number of second image patches with a similarity greater than the first preset similarity as the second number corresponding to the first image patch, so as to obtain the second data corresponding to each first image patch corresponding to the object. According to the second numbers corresponding to each first image patch, determine that the second object is the same second object as the object.

[0173] Optionally, determining that the second object is the same second object as the object according to the second numbers corresponding to each first image patch includes: calculating the sum of the second numbers corresponding to all first image patches to obtain a similarity number. When the similarity number is greater than the preset similarity number, it indicates that the similarity between the object and the second object is relatively high, then continue to determine whether the two are the same target, and determine the distance difference between the center point of the human detection box of the object and the center point of the human detection box of the second object. Among them, the distance difference includes a width difference and / or a height difference. According to the distance difference, determine that the second object is the same second object as the object.

[0174] Specifically, since the time difference between the second image and the first image is very short, the movement amplitude of the object is small, and the distance difference between the object and the second object is small. When the distance difference is greater than the preset distance difference, it indicates that the distance between the object and the second object is large, and the two do not belong to the same target, so it is determined that the second object is different from the object. When the distance difference is less than or equal to the preset distance difference, it indicates that the distance between the object and the second object is small, and the two belong to the same target, so it is determined that the second object is different from the object.

[0175] Optionally, the distance difference is calculated from the coordinate values of the center points of the human detection frames corresponding to the object and the second object. Among them, when the distance difference includes a width difference value, the width difference value indicates the difference between the two in the image width direction. Correspondingly, the preset distance difference includes a preset width distance difference (for example, 0.5*W); when the distance difference includes a height difference value, the height difference value indicates the difference between the two in the image height direction. Correspondingly, the preset distance difference includes a preset height distance difference (for example, 0.5*H).

[0176] Among them, W represents the image width, for example, the image width of the first image. H represents the image height, for example, the image height of the first image.

[0177] Optionally, when the similarity number is less than or equal to the preset similarity number, it indicates that the similarity between the object and the second object is low, so it is determined that the object and the second object do not belong to the same target.

[0178] Optionally, based on the person detection module, the above process of determining whether the object and the second object belong to the same target can be executed. Of course, the person detection module model can also perform human detection to determine the human detection frame.

[0179] Among them, the person detection model is based on the yolov5 model with a multi-branch structure and loads multiple datasets with different weights at the same time. By training different datasets using the method of cross-mosaicing, an improved yolov5 model, that is, the person detection model, is obtained.

[0180] Step 304, when the target classification result of the object indicates that the type of the object is the first preset type, determine the accompanying person detection result corresponding to the object according to the first object other than the object in the first image. Among them, the first preset type represents the object type that provides specific services.

[0181] Among them, the accompanying person detection result corresponding to the object indicates whether there is an accompanying person for the object.

[0182] In this embodiment, for each object in the first image, when the target classification result of the object indicates that the type of the object is the first preset type, it indicates that the object is a special person, and it is necessary to determine whether it is necessary for the staff to provide specific services for the object. Then, it is necessary to continue to determine whether there is an accompanying person for the object to determine whether it is necessary to provide services for the object.

[0183] Among them, the first preset type includes at least one of the disabled type, the pregnant woman type, and the elderly type. For example, when the first preset type includes the disabled type, as long as the target classification result of the object indicates that the object is disabled, it is considered that the target classification result of the object indicates that the type of the object is the first preset type; another example is that when the first preset type includes the disabled type and the pregnant woman type, when the target classification result of the object indicates that the object is both disabled and pregnant, it is considered that the target classification result of the object indicates that the type of the object is the first preset type.

[0184] Optionally, determining the accompanying person detection result corresponding to the object according to the first object other than the object in the first image includes:

[0185] Based on the human detection box corresponding to the object and the human detection boxes corresponding to each first object, determine the distance between the object and each first object.

[0186] In the case where the distances between the object and each first object are all greater than the preset accompanying distance, it is determined that the accompanying person detection result corresponding to the object indicates that there is no accompanying person for the object.

[0187] In the case where there is a distance between the object and the first object that is less than or equal to the preset accompanying distance, it is determined that the accompanying person detection result corresponding to the object indicates that there is an accompanying person for the object.

[0188] Specifically, since the accompanying person of a person is generally relatively close to him / her, while the distance between strangers is relatively far, therefore, the distance between two objects can be used to determine whether the two objects are together, that is, whether one object is the accompanying person of the other object.

[0189] When the target classification result of the object in the first image indicates that the type of the object is the first preset type, the objects in the first image other than the object are taken as the first objects. Calculate the distances between the object and each of the first objects. When the distance between the object and a first object is greater than the preset accompanying distance, it indicates that the two are far apart and the first object is not the accompanying person of the object. When all the first objects are not the accompanying persons of the object, it indicates that the object is handling business alone and specific services need to be provided for it. Then, it is determined that the accompanying person detection result corresponding to the object indicates that the object has no accompanying person. When the distance between the object and a first object is less than or equal to the preset accompanying distance, it indicates that the two are close and the first object is the accompanying person of the object, that is, it indicates that the object is not handling business alone. Although the object is a special person, the staff does not need to provide services for it. Then, it is determined that the accompanying person detection result corresponding to the object indicates that the object has an accompanying person.

[0190] Step 305: Output the first information according to the accompanying person detection result corresponding to the object.

[0191] In this embodiment, when the target classification result of the object indicates that the type of the object is the first preset type, it indicates that the object is a special person who needs specific services. When the accompanying person detection result corresponding to the object indicates that the object has an accompanying person, it indicates that the accompanying person can help the object handle the business and the staff does not need to provide specific services. Then, there is no need to output the first information, that is, continue to determine whether other objects in the first image need the staff to provide services or continue to process other images to determine whether the objects in other images need the staff to provide services, so that the staff can serve other objects with needs in time and avoid waste of resources.

[0192] When the accompanying person detection result corresponding to the object indicates that the object has no accompanying person, it indicates that the object is handling business alone. To improve the service quality, the first information is output to prompt the relevant staff to provide specific services (such as love services) for the object.

[0193] Optionally, outputting the first information according to the accompanying person detection result corresponding to the object includes:

[0194] In the case where the accompanying person detection result corresponding to the object indicates that the object has no accompanying person, determine the target terminal corresponding to the object.

[0195] Output the first information to the target terminal. The first information includes a prompt message and / or control information. The prompt message is used to be displayed on the target terminal to prompt to provide specific services. The control information is used to control the target terminal to perform specific service reminders in accordance with a preset reminder method.

[0196] Specifically, when the first information includes a prompt message, the prompt message is sent to the target terminal so that the target terminal displays the prompt message, thereby prompting the staff corresponding to the target terminal to provide specific services to the object; when the first information includes control information, the control information is sent to the target terminal to control the target terminal to operate according to a preset reminder method, thereby reminding the staff corresponding to the target terminal to provide specific services to the object.

[0197] Among them, the target terminal includes a service desk, a service bell, etc. When the target terminal is a service bell, it gives an audible and visual reminder when receiving the control information.

[0198] Optionally, when determining the target terminal corresponding to the object, the target terminal closest to the object can be selected, that is, the distances between the object and each terminal are obtained, and the terminal with the shortest distance is used as the target terminal corresponding to the object, so that the staff corresponding to the target terminal can quickly provide services to the object and ensure the timeliness of the service.

[0199] Optionally, when determining the distances between the object and each terminal, it can be determined based on the position coordinates of the object and the position coordinates corresponding to each terminal.

[0200] Specifically, when calculating the distances between the object and each terminal, relevant calibration algorithms can be used to perform coordinate system conversion, so that the coordinate system where the position coordinates of the object are located is consistent with the coordinate system where the position coordinates of the terminal are located.

[0201] To better provide services, the identity of the object can be determined through a preset information library to determine the relevant information of the object, that is, face detection is performed on the image area corresponding to the human detection frame to obtain the face detection frame corresponding to the object, and based on the face recognition algorithm, the face detection frame corresponding to the object is matched with the face information in the preset information library. When the face detection frame corresponding to the object matches the face information successfully, the prompt message is determined according to the record information corresponding to the face information in the preset information library. Among them, the prompt message includes the object identifier and / or historical business handling information. The historical business handling information indicates the business that the object has handled.

[0202] Among them, the face information includes the face image of the object (i.e., the historical object) stored in the preset information library, and / or the face feature data of the historical object. The historical object is the person who handled business at the place corresponding to the target area before the current moment. When the face information includes a face image, the feature data corresponding to the face image is extracted to obtain the face feature data corresponding to the historical object corresponding to the face information.

[0203] Specifically, based on the face recognition algorithm, extract the face feature data corresponding to the face detection frame of the object, and based on the face feature data corresponding to the object and the face feature data corresponding to each historical object, match the object with each historical object. When there is historical object corresponding face feature data that is consistent with or has a high similarity to the face feature data corresponding to the object, it indicates that the historical object matches the object, that is, the two belong to the same target, then obtain the record information corresponding to the historical object and generate a prompt message including the record information.

[0204] Among them, the record information corresponding to the historical object includes the object identifier corresponding to the historical object, the type of the historical object, and the historical business handling information.

[0205] Among them, the object identifier includes the name of the object, the ID (Identity document) number, etc.

[0206] Optionally, when the target classification result of the object indicates that the type of the object is inconsistent with the type of the historical object that matches the object, then update the type of the historical object in the preset information library.

[0207] Optionally, when the face feature data corresponding to all historical objects is inconsistent with or has a high similarity to the face feature data corresponding to the object, it indicates that all historical objects do not match the object, that is, it indicates that the object has not handled business at the place corresponding to the target area before, then generate the record information corresponding to the object and add it to the preset information library for subsequent determination of the body.

[0208] Among them, the record information corresponding to the object includes the newly assigned object identifier for the object, the target classification result of the object, that is, the type of the object, etc. When the object handles business, add the business information handled by the object to the record information.

[0209] Match the face detection frame corresponding to the object with the face information corresponding to the historical object in the preset information library to identify the identity of the object, that is, determine whether there is a historical object that belongs to the same object as the object.

[0210] Optionally, in order to improve the accuracy of identifying the identity of the object, perform face alignment processing on the face detection frame corresponding to the object, that is, adjust the face to be upright. After performing face alignment processing on the object, determine the image quality information corresponding to the aligned face detection frame. When it is determined that the image quality information meets the preset quality conditions, match the aligned face detection frame with the face information in the preset information library to further improve the accuracy of identifying the identity of the object.

[0211] Among them, when performing face alignment processing, face alignment can be performed based on a face key point model. Specifically, a 5-point face key point model is used to find the eyes and mouth, and the positions of the center points of the left eye, right eye, and mouth of the human eyes are calculated, and their positions are (a1, a2), (a3, a4), (a5, a6) respectively. The coordinates of the center points of the aligned face eyes are set as (A1, A2), (A3, A4), (A5, A6). The transformation coefficients are obtained using the affine transformation formula, and the obtained coefficients are x, y, z, w, p, t respectively; finally, the original image, that is, the image area corresponding to the face detection frame of the object, is obtained based on the transformation coefficients, and the aligned image, that is, the aligned face detection frame, is obtained.

[0212] Among them, the image quality information includes one or more of the image resolution, the pitch angle of the face corresponding to the object, and the image illumination intensity; correspondingly, the preset quality conditions include one or more of the image resolution being greater than a preset resolution (for example, 50*50), the pitch angle of the face being less than a preset angle value (for example, 35 degrees), and the image illumination intensity being within a preset intensity range. For example, when the image quality information includes the image resolution and the pitch angle of the face, correspondingly, the preset quality conditions include the image resolution being greater than the preset resolution and the pitch angle of the face being less than the preset angle value. If the image resolution is greater than the preset resolution and the pitch angle of the face is less than the preset angle value, it is determined that the image quality information, that is, the face quality corresponding to the object meets the preset quality conditions. If the image resolution is less than or equal to the preset resolution, or the pitch angle of the face is greater than or equal to the preset angle value, it is determined that the image quality information, that is, the face quality corresponding to the object does not meet the preset quality conditions.

[0213] Among them, the image illumination intensity indicates the illumination intensity of the image area corresponding to the face detection frame of the object.

[0214] Optionally, the aligned face detection frame corresponding to the object can also be determined through a face quality model, that is, whether the face quality corresponding to the object meets the preset quality conditions, that is, to determine whether the face quality score corresponding to the object is greater than a preset score value. Specifically, the image size is set, the image is grayscale, and the image area corresponding to the face detection frame of the object is sequentially passed through the Tenengrad gradient function, Brenner gradient function, Laplacian gradient function, and SMD (gray variance) function for face quality judgment to obtain the face quality score. When the face quality score is greater than the preset score value, it is determined that the face quality corresponding to the object meets the requirements; otherwise, it is determined that the face quality corresponding to the object does not meet the requirements.

[0215] Optionally, when the image quality information corresponding to the object does not meet the preset quality conditions, continue to process other objects in the first image, or process the next frame of image.

[0216] In this embodiment, when the target classification result of an object indicates that the type of the object is the first preset type, that is, when it is determined that the object is a special person who needs specific services, it is determined whether staff need to provide specific services to the object by judging whether there is an accompanying person for the object. When there is no accompanying person for the object, the first information is output so that the staff can provide specific services to the object in a timely manner, improving the service quality and also avoiding waste of resources.

[0217] Corresponding to the embodiment of the foregoing method, this specification also provides an embodiment of a device and a terminal to which the device is applied.

[0218] The embodiment of the service providing device in this specification can be applied to an electronic device, such as a server or a terminal device. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful device, it is formed by the processor where it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for operation. From the hardware level, as Figure 5 shown, it is a hardware structure diagram of the electronic device where the service providing device in the embodiment of this specification is located. In addition to Figure 5 the processor 510, memory 530, network interface 520, and non-volatile memory 540 shown, the electronic device where the service providing device 531 in the embodiment is located usually includes other hardware according to the actual functions of the electronic device, which will not be elaborated here.

[0219] As Figure 6 shown, Figure 6 is a block diagram of a service providing device shown according to an exemplary embodiment of this specification. The device includes:

[0220] An image acquisition module 610, configured to acquire a first image of a target area;

[0221] A detection module 620, configured to perform human body detection on the first image to obtain at least one human body detection result, and the human body detection result includes a human body detection frame;

[0222] A classification module 630, configured to classify the image area corresponding to the human body detection frame in multiple dimensions, and obtain the target classification result of the object corresponding to the human body detection frame according to the classification results of each dimension;

[0223] A processing module 640, configured to output first information according to the target classification result of the object; wherein, the first information indicates to provide specific services.

[0224] Optionally, the target classification result indicates the type of the object;

[0225] The processing module 640 is specifically configured to:

[0226] When the target classification result of the object indicates that the type of the object is the first preset type, determine the accompanying person detection result corresponding to the object according to the first object other than the object in the first image; wherein, the first preset type represents the type of the object providing a specific service;

[0227] Output the first information according to the accompanying person detection result corresponding to the object.

[0228] Optionally, the processing module 640 is further configured to:

[0229] Determine the distances between the object and each of the first objects based on the human detection frame corresponding to the object and the human detection frames corresponding to each of the first objects;

[0230] When the distances between the object and each of the first objects are all greater than the preset accompanying distance, determine that the accompanying person detection result corresponding to the object indicates that there is no accompanying person for the object;

[0231] When there is a distance between the object and a first object that is less than or equal to the preset accompanying distance, determine that the accompanying person detection result corresponding to the object indicates that there is an accompanying person for the object.

[0232] Optionally, the processing module 640 is further configured to:

[0233] When the accompanying person detection result corresponding to the object indicates that there is no accompanying person for the object, determine the target terminal corresponding to the object;

[0234] Output the first information to the target terminal; wherein, the first information includes a prompt message and / or control information; the prompt message is used to be displayed on the target terminal to prompt for providing a specific service; the control information is used to control the target terminal to perform a specific service reminder according to a preset reminder method.

[0235] Optionally, the processing module 640 is further configured to:

[0236] Obtain the distances between the object and each terminal, and use the terminal with the shortest distance as the target terminal corresponding to the object.

[0237] Optionally, the detection module 620 is further configured to:

[0238] Perform face detection on the image region corresponding to the human detection frame to obtain the face detection frame corresponding to the object, and based on the face recognition algorithm, match the face detection frame corresponding to the object with the face information in the preset information database;

[0239] When the face detection box corresponding to the object successfully matches the face information, determine the prompt information according to the record information corresponding to the face information in the preset information library; wherein, the prompt information includes the object identifier and / or historical business handling information; the historical business handling information indicates the business that the object has handled.

[0240] Optionally, the detection module 620 is further configured to:

[0241] Perform face alignment processing on the face detection box;

[0242] Determine the image quality information corresponding to the aligned face detection box;

[0243] When it is determined that the image quality information meets the preset quality conditions, match the aligned face detection box with the face information in the preset information library.

[0244] Optionally, the first image is a frame in the image sequence; the image sequence includes a preset number of continuously captured images;

[0245] The classification module 630 is further configured to:

[0246] For any dimension, determine whether there is a second object identical to the object in the second image; wherein, the second image is any frame other than the first image in the image sequence; the first type indicates the type in the dimension;

[0247] When there is a second object identical to the object, obtain the classification result of the second object in the dimension;

[0248] Obtain the first number of classification results indicating that the first type of the second object is the second preset type;

[0249] When the first number is less than the preset number, adjust the classification result of the object in the dimension.

[0250] Optionally, the classification module 630 is further configured to:

[0251] Based on the image region corresponding to the human body detection box corresponding to the object, determine the feature data corresponding to the object, and obtain the feature data corresponding to each second object in the second image;

[0252] For each second object, determine the Euclidean distance between the object and the second object according to the feature data corresponding to the object and the feature data corresponding to the second object;

[0253] When the Euclidean distance is less than the preset distance threshold, determine that the second object is a second object identical to the object.

[0254] Optionally, the classification module 630 is further configured to:

[0255] When the Euclidean distance is less than a preset distance threshold, crop the image region corresponding to the human detection box of the object to obtain a plurality of first image patches, and crop the image region corresponding to the human detection box of the second object to obtain a plurality of second image patches;

[0256] For each first image patch, determine the similarity between the first image patch and each second image patch respectively, and determine the number of second image patches whose similarity is greater than a first preset similarity;

[0257] Take the number of second image patches whose similarity is greater than the first preset similarity as the second number corresponding to the first image patch;

[0258] According to the second numbers corresponding to each first image patch, determine that the second object is the same second object as the object.

[0259] Optionally, the classification module 630 is further configured to:

[0260] Calculate the sum of the second numbers corresponding to all the first image patches to obtain a similarity number;

[0261] When the similarity number is greater than a preset similarity number, determine the distance difference between the center point of the human detection box of the object and the center point of the human detection box of the second object; wherein, the distance difference includes a width difference value and / or a height difference value;

[0262] According to the distance difference, determine that the second object is the same second object as the object.

[0263] Optionally, the dimension includes one or more of a disabled person dimension, a pregnant woman dimension, and an elderly person dimension;

[0264] The classification module 630 is specifically configured to:

[0265] Use a first preset classification model to perform disabled person classification based on the image region corresponding to the human detection box to obtain a classification result corresponding to the disabled person dimension. Wherein, the first preset recognition model is used to recognize whether the object corresponding to the human detection box belongs to a disabled person.

[0266] And / or,

[0267] Use a second preset classification model to perform pregnant woman classification based on the image region corresponding to the human detection box to obtain a classification result corresponding to the pregnant woman dimension. Wherein, the second preset classification model is used to recognize whether the object corresponding to the human detection box belongs to a pregnant woman.

[0268] And / or,

[0269] Perform face detection on the image region corresponding to the human detection box to obtain a face detection box.

[0270] Using a third preset classification model, classify the image region corresponding to the face detection box for the elderly to obtain the classification result corresponding to the elderly dimension. Among them, the third preset classification model is used to identify whether the object corresponding to the face detection box belongs to the elderly.

[0271] Correspondingly, this specification also provides an electronic device, the device includes a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to:

[0272] Obtain a first image of the target area;

[0273] Perform human detection on the first image to obtain at least one human detection result, and the human detection result includes a human detection box;

[0274] Classify the image region corresponding to the human detection box in multiple dimensions, and obtain the target classification result of the object corresponding to the human detection box according to the classification results of each dimension;

[0275] Output first information according to the target classification result of the object; wherein, the first information indicates to provide a specific service.

[0276] The implementation processes of the functions and roles of each module in the above device are specifically described in the implementation processes of the corresponding steps in the above method, and will not be elaborated here.

[0277] In another embodiment, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the service providing method as described above is implemented.

[0278] In another embodiment, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the service providing method as described above is implemented.

[0279] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiments described above are only illustrative, and the modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place, or may be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this specification. Those of ordinary skill in the art can understand and implement it without creative work.

[0280] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0281] Those skilled in the art will readily conceive of other implementations of this specification after considering the specification and practicing the invention claimed herein. This specification is intended to cover any variations, uses, or adaptations of this specification, which follow the general principles of this specification and include known common knowledge or conventional technical means in the technical field not claimed in this application. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this specification are pointed out by the following claims. Additionally, the data involved in this application may be data authorized by the user or fully authorized by all parties.

[0282] It should be understood that this specification is not limited to the exact structures described above and shown in the figures, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.

[0283] The above are only the preferred embodiments of this specification and are not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification shall be included within the scope of protection of this specification.

Claims

1. A service providing method, characterized in that, Including: Obtain a first image of a target area; Perform human detection on the first image to obtain at least one human detection result, where the human detection result includes a human detection box; Classify the image area corresponding to the human detection box in multiple dimensions, and obtain a target classification result of the object corresponding to the human detection box according to the classification results of each dimension; Output first information according to the target classification result of the object; wherein, the first information indicates to provide a specific service; Wherein, the first image is a frame in an image sequence; the image sequence includes a preset number of continuously captured images; The method further includes: For any dimension, determine whether there is a second object in a second image that is the same as the object; wherein, the second image is any frame in the image sequence other than the first image; the type of the second object in the dimension is indicated by a first type; When there is a second object that is the same as the object, obtain the classification result of the second object in the dimension; Obtain a first number of classification results indicating that the first type of the second object is a second preset type, where the second preset type indicates the type of the object that requires a specific service corresponding to the dimension; When the first number is less than a preset number, adjust the classification result of the object in the dimension.

2. The method according to claim 1, wherein The target classification result indicates the type of the object; The outputting of the first information according to the target classification result of the object includes: When the target classification result of the object indicates that the type of the object is a first preset type, determine a companion detection result corresponding to the object according to a first object other than the object in the first image; wherein, the first preset type represents the type of the object that provides a specific service; Output first information according to the companion detection result corresponding to the object.

3. The method according to claim 2, wherein The determining of the companion detection result corresponding to the object according to the first object other than the object in the first image includes: Based on the human detection box corresponding to the object and the human detection boxes corresponding to each first object, determine the distance between the object and each first object; When the distance between the object and each first object is greater than a preset companion distance, determine that the companion detection result corresponding to the object indicates that the object has no companion; When there is a distance between an object and the first object that is less than or equal to the preset companion distance, determine that the companion detection result corresponding to the object indicates that the object has a companion.

4. The method according to claim 2, characterized in that, The outputting of the first information according to the companion detection result corresponding to the object includes: When the companion detection result corresponding to the object indicates that the object has no companion, determine a target terminal corresponding to the object; Output first information to the target terminal; wherein, the first information includes a prompt message and / or control information; the prompt message is used to be displayed on the target terminal to prompt to provide a specific service; the control information is used to control the target terminal to perform a specific service reminder in a preset reminder manner.

5. The method according to claim 4, wherein The method further includes: Perform face detection on the image region corresponding to the human detection box to obtain the face detection box corresponding to the object, and based on the face recognition algorithm, match the face detection box corresponding to the object with the face information in the preset information database; When the face detection box corresponding to the object matches the face information successfully, determine the prompt information according to the record information corresponding to the face information in the preset information database; wherein, the prompt information includes an object identifier and / or historical service handling information; the historical service handling information indicates the services that the object has handled.

6. The method according to claim 5, wherein The matching of the face detection box corresponding to the object with the face information in the preset information database includes: Perform face alignment processing on the face detection box; Determine the image quality information corresponding to the aligned face detection box; When it is determined that the image quality information meets the preset quality conditions, match the aligned face detection box with the face information in the preset information database.

7. The method according to claim 1, characterized in that The determination of whether there is a second object in the second image that is the same as the object includes: Based on the image region corresponding to the human detection box corresponding to the object, determine the feature data corresponding to the object, and obtain the feature data corresponding to each second object in the second image; For each second object, determine the Euclidean distance between the object and the second object according to the feature data corresponding to the object and the feature data corresponding to the second object; When the Euclidean distance is less than the preset distance threshold, determine that the second object is the same second object as the object.

8. The method according to claim 7, wherein The determination that the second object is the same second object as the object when the Euclidean distance is less than the preset distance threshold includes: When the Euclidean distance is less than the preset distance threshold, crop the image region corresponding to the human detection box corresponding to the object to obtain a plurality of first image blocks, and crop the image region corresponding to the human detection box corresponding to the second object to obtain a plurality of second image blocks; For each first image block, determine the similarity between the first image block and each second image block respectively, and determine the number of second image blocks with a similarity greater than the first preset similarity; Use the number of second image blocks with a similarity greater than the first preset similarity as the second number corresponding to the first image block; According to the second numbers corresponding to each first image block, determine that the second object is the same second object as the object.

9. The method according to claim 8, characterized in that, The determination that the second object is the same second object as the object according to the second numbers corresponding to each first image block includes: Calculate the sum of the second numbers corresponding to all first image blocks to obtain the similarity number; When the similarity number is greater than the preset similarity number, determine the distance difference between the center point of the human detection box of the object and the center point of the human detection box of the second object; wherein, the distance difference includes a width difference and / or a height difference; According to the distance difference, determine that the second object is the same second object as the object.

10. The method according to any one of claims 1 to 6, characterized in that, The dimensions include one or more of the disabled dimension, the pregnant woman dimension, and the elderly dimension; The classification of the image region corresponding to the human detection box into multiple dimensions includes: Using a first preset classification model, performing disabled person classification based on the image region corresponding to the human detection box to obtain a classification result corresponding to the disabled person dimension; wherein, the first preset recognition model is used to identify whether the object corresponding to the human detection box belongs to a disabled person; and / or, Using a second preset classification model, performing pregnant woman classification based on the image region corresponding to the human detection box to obtain a classification result corresponding to the pregnant woman dimension; wherein, the second preset classification model is used to identify whether the object corresponding to the human detection box belongs to a pregnant woman; and / or, Performing face detection on the image region corresponding to the human detection box to obtain a face detection box; Using a third preset classification model, performing elderly person classification on the image region corresponding to the face detection box to obtain a classification result corresponding to the elderly dimension; wherein, the third preset classification model is used to identify whether the object corresponding to the face detection box belongs to an elderly person.

11. A service providing device, characterized in that, Including: An image acquisition module, configured to acquire a first image of a target region; A detection module, configured to perform human detection on the first image to obtain at least one human detection result, where the human detection result includes a human detection box; A classification module, configured to classify the image region corresponding to the human detection box into multiple dimensions, and obtain a target classification result of the object corresponding to the human detection box according to the classification results of each dimension; A processing module, configured to output first information according to the target classification result of the object; wherein, the first information indicates to provide a specific service; Wherein, the first image is a frame in an image sequence; the image sequence includes a preset number of continuously captured images; The classification module is further configured to: For any dimension, determine whether there is a second object in a second image that is the same as the object; wherein, the second image is any frame in the image sequence other than the first image; the type of the second object in the dimension is indicated by a first type; In the case where there is a second object that is the same as the object, obtain the classification result of the second object in the dimension; Obtain a first number of classification results indicating that the first type of the second object is a second preset type, where the second preset type indicates the type of the object corresponding to the dimension that requires a specific service; In the case where the first number is less than a preset number, adjust the classification result of the object in the dimension.

12. A computer device, characterized in that, Including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, when the processor executes the program, the service providing method according to any one of claims 1 to 10 is implemented.

13. A computer-readable storage medium, characterized in that, Computer-executable instructions are stored in the computer-readable storage medium, and when the processor executes the computer-executable instructions, the service providing method according to any one of claims 1 to 10 is implemented.

Images