Article rental state detection method, device and equipment and storage medium

Abstract

The invention discloses an article lease state detection method, device and equipment and a storage medium, which can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic and auxiliary driving, and the method further increases an FOV to better capture a detection object by means of the image acquisition capability of an identity recognition device after a lease operation is triggered, thereby improving the detection efficiency. The position change information of the detection object is determined through image analysis, so that the item renting state of the target object for the rented item is determined, for example, whether the first rented item allocated to the target object is successfully unlocked or not and whether the target object successfully takes out the first rented item or not are determined; therefore, on the basis of not increasing hardware conditions, the article rental state detection is realized, the article rental state sensing accuracy is improved, so that relevant remedial measures or alarms can be taken in time when the rental state is abnormal, the possibility that a leasee cannot use the rental article is reduced, and the service quality of article rental is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and provides a method, device, equipment and storage medium for detecting the rental status of an item. Background Art

[0002] Currently, item rental has brought great convenience to daily life, and the application scenarios are becoming increasingly rich; for example, power bank rental (which can also be called shared power bank), bicycle rental (which can also be called shared bicycle), baby carriage rental (which can also be called shared baby carriage), etc.

[0003] Taking power bank rental as an example, when renting a power bank, it is usually necessary to control the item rental device to eject the power bank for the renter to take after identifying the identity of the renter. However, due to the hardware aging of the item rental device or problems with the device structure, abnormal problems such as unsuccessful ejection of the power bank may occur. If the item rental device fails to detect these abnormalities and thinks that the power bank has been successfully ejected, it will cause the renter to be unable to use the power bank, thus affecting the normal use of the power bank by the renter.

[0004] Obviously, the hardware problems of the item rental device under the related technology will affect the service quality of item rental. Summary of the Invention

[0005] Embodiments of the present application provide a method, device, equipment and storage medium for detecting the rental status of an item, which are used to assist in detecting the rental status of an item based on an identity recognition device, improve the accuracy of perceiving the rental status of an item, and thus improve the service quality of item rental.

[0006] On the one hand, a method for detecting the rental status of an item is provided. The method includes:

[0007] In response to a rental operation triggered by a target object, adjusting the field of view angle of an identity recognition device associated with the item rental device from a first field of view angle to a second field of view angle, where the identity recognition device is used to perform identity recognition based on the collected image, and the second field of view angle is greater than the first field of view angle;

[0008] Obtaining a set of status detection images collected at the second field of view angle after the identity recognition device recognizes the identity information of the target object, where the set of status detection images includes at least one detection object presented in the status detection images, and the at least one detection object includes a first rental item assigned to the target object;

[0009] Determining the position change information of each of the at least one detection object between the status detection images in the set of status detection images according to the set of status detection images;

[0010] Determine the item rental status of the target object for the first rental item according to the position change information corresponding to each of the at least one detection object.

[0011] On the one hand, an item rental status detection device is provided, and the device includes:

[0012] An adjustment unit, configured to adjust the field of view angle of an identity recognition device associated with an item rental device from a first field of view angle to a second field of view angle in response to a rental operation triggered by a target object, where the identity recognition device is used for identity recognition according to the collected images, and the second field of view angle is greater than the first field of view angle;

[0013] An image acquisition unit, configured to obtain a set of status detection images acquired at the second field of view angle after the identity recognition device recognizes the identity information of the target object, where the set of status detection images includes at least one detection object presented in the status detection images, and the at least one detection object includes a first rental item assigned to the target object;

[0014] An image processing unit, configured to determine the position change information of each of the at least one detection object between the status detection images in the set of status detection images according to the set of status detection images;

[0015] A status determination unit, configured to determine the item rental status of the target object for the first rental item according to the position change information corresponding to each of the at least one detection object.

[0016] In a possible implementation manner, the position change information includes a first actual movement trajectory of the first rental item, and the item rental status includes an unlocking status of the first rental item; then the status determination unit is specifically configured to:

[0017] If the first actual movement trajectory matches a preset first reference movement trajectory, determine that the unlocking status of the first rental item is successful unlocking;

[0018] If the first actual movement trajectory does not match the first reference movement trajectory, determine that the unlocking status of the first rental item is failed unlocking.

[0019] In a possible implementation manner, the set of status detection images includes a first image subset, and the first image subset includes a reference image and an image sequence after the image acquisition moment and located after the reference image; then the image processing unit is specifically configured to:

[0020] Determine the original position of the first rental item in the reference image according to the first item information corresponding to the first rental item;

[0021] Determine the position sequence of the first leased item in the image sequence according to the first item information and the original position, where the position sequence includes the actual positions of the first leased item in each image of the image sequence;

[0022] Determine the first actual movement trajectory according to the original position and the position sequence.

[0023] In a possible implementation, the item lease status includes the unlocking status of the first leased item, the status detection image set includes a first image subset, and the first image subset includes a reference image and an image sequence after the image acquisition moment of the reference image; then the status determination unit is specifically configured to:

[0024] Determine the original position of the first leased item in the reference image according to the first item information corresponding to the first leased item;

[0025] Extract the reference pixel features corresponding to the original position from the reference image;

[0026] Extract the target pixel features corresponding to the original position from each image in the image sequence respectively;

[0027] Determine the unlocking status according to the reference pixel features and the feature differences between the obtained target pixel features.

[0028] In a possible implementation, the position change information further includes the second actual movement trajectory of the limb of the target object, and the item lease status includes the taken-out status of the first leased item; then the status determination unit is further specifically configured to:

[0029] If the second actual movement trajectory meets the preset conditions, determine that the taken-out status of the first leased item is taken out successfully;

[0030] If the second actual movement trajectory does not meet the preset conditions, determine that the unlocking status of the first leased item is taken out failed;

[0031] Wherein, the preset conditions include: the second actual movement trajectory matches the first actual movement trajectory, and the distance between the limb and the first leased item in each image corresponding to the second actual movement trajectory is not greater than the preset threshold.

[0032] In a possible implementation, the status detection image set includes a second image subset, and the item lease status includes the taken-out status of the first leased item; then the status determination unit is further specifically configured to:

[0033] Input each image included in the second image subset into the trained image recognition model to obtain the image recognition result corresponding to each image, where the image recognition result is used to indicate the interaction type between the limb and the first leased item in the corresponding image;

[0034] Determine the removal status based on the obtained image recognition results of each image;

[0035] Among them, the status detection model is trained based on multiple training samples. Each training sample in the multiple training samples includes a sample image and its corresponding training label, and the training label is used to indicate the interaction type between the limb and the rental item.

[0036] In a possible implementation manner, the status detection image set includes a second image subset, and the item rental status includes the removal status of the first rental item; then the status determination unit is further specifically configured to:

[0037] For each status detection image included in the second image subset, perform the following operations respectively:

[0038] For a status detection image, input the status detection image into the trained image segmentation model to obtain an image segmentation result corresponding to the status detection image. The image segmentation result includes the position information of the limb of the target object and the first rental item in the status detection image respectively;

[0039] Determine the relative position information between the limb and the first rental item according to the position information of the limb and the first rental item respectively;

[0040] Determine the removal status according to the relative position information corresponding to each status detection image and the preset position constraint conditions.

[0041] In a possible implementation manner, the device further includes an execution unit, configured to:

[0042] If the item rental status is unlocking failure or removal failure, perform at least one of the following operations:

[0043] Reassign a second rental item for the target object from the rental items in the idle state;

[0044] Generate an alarm message based on the item rental status and send the alarm message to the background server.

[0045] In a possible implementation manner, the adjustment unit is specifically configured to:

[0046] In response to the rental operation, collect an identity recognition image of the target object at the first field of view angle;

[0047] If the identity information of the target object is successfully recognized according to the identity recognition image, send an adjustment instruction to the identity recognition device, and the adjustment instruction is used to instruct the identity recognition device to adjust the field of view angle to the second field of view angle.

[0048] In a possible implementation, the identity recognition device is a palm-sweeping device, and the identity recognition image is a palm image. Then, the image processing unit is further configured to:

[0049] Extract features from the palm image to obtain the target palm features of the target object;

[0050] Match the target palm features with at least one reference palm feature to determine the reference palm feature that matches the target palm features;

[0051] Determine the identity information associated with the reference palm feature as the identity information of the target object.

[0052] On the one hand, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of any of the above methods are implemented.

[0053] On the one hand, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0054] On the one hand, a computer program product is provided. The computer program product includes a computer program, which is stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the steps of any of the above methods.

[0055] In the embodiments of the present application, when the target object triggers a rental operation, the field of view (FOV) of the identity recognition device can be increased to collect a status detection image corresponding to the detection object. Then, according to the position change information of the detection object in the status detection image, the rental status of the item is determined. It can be seen that this method utilizes the image acquisition ability of the identity recognition device. After triggering the rental operation, the FOV is further increased to better capture the detection object, and the position change information of the detection object is determined through image analysis, so as to determine the item rental status of the target object for the rental item. For example, whether the first rental item allocated to the target object is successfully unlocked and whether the target object successfully takes out the first rental item, etc. Thus, the detection of the item rental status is realized without adding hardware conditions, improving the accuracy of item rental status perception, so that relevant remedial measures or alarms can be taken in time when the rental status is abnormal, reducing the possibility that the lessee cannot use the rental item, and improving the service quality of item rental. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] To more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings required for use in the description of the embodiments or the related art. Obviously, the drawings in the following description are only those of the embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0057] Figure 1 Schematic diagram of the application scenario provided by the embodiment of the present application;

[0058] Figure 2 Schematic diagram of the system architecture provided by the embodiment of the present application;

[0059] Figure 3 Schematic flowchart of the method for detecting the rental status of items provided by the embodiment of the present application;

[0060] Figure 4 Schematic diagram of adjusting the FOV provided by the embodiment of the present application;

[0061] Figure 5 Schematic flowchart of the process for identifying identity information by palm brushing provided by the embodiment of the present application;

[0062] Figure 6 Schematic diagram of the status judgment related to the item rental process provided by the embodiment of the present application;

[0063] Figure 7 Schematic diagram of the process for detecting the rental status of a power bank provided by the embodiment of the present application;

[0064] Figure 8A and Figure 8B Schematic diagram of the trajectory provided by the embodiment of the present application;

[0065] Figure 9 Schematic flowchart of the process for detecting the rental status of a power bank provided by the embodiment of the present application;

[0066] Figure 10 Schematic diagram of a structure of the device for detecting the rental status of items provided by the embodiment of the present application;

[0067] Figure 11 Schematic diagram of the composition structure of the computer device provided by the embodiment of the present application;

[0068] Figure 12 Schematic diagram of the composition structure of another computer device provided by the embodiment of the present application. Detailed implementation manners

[0069] To make the objectives, technical solutions, and advantages of this application more clearly understood, the following will describe the technical solutions in the embodiments of this application clearly and completely in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application. Without conflict, the embodiments in this application and the features in the embodiments can be combined arbitrarily with each other. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0070] It can be understood that in the following specific embodiments of this application, user data may be involved, such as user voice data. Then, when the embodiments of this application are applied to specific products or technologies, relevant permissions or consents need to be obtained, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0071] To facilitate the understanding of the technical solutions provided by the embodiments of this application, some key terms used in the embodiments of this application are explained here first:

[0072] Palm brushing operation: Palm brushing recognition is a technology that exchanges identity information through palm multimedia information. The palm brushing operation refers to the operation that a user performs to provide the information required for identity verification when a business needs to be verified and the palm brushing recognition method is used for identity verification. Usually, it means placing one's own palm within the area that can be collected by the palm brushing device. Generally speaking, the palm brushing device can include an image acquisition device to collect palm images for subsequent identity recognition processes. For example, it can include a 3D camera. The 3D camera is analogous to a traditional camera and adds software and hardware related to liveness, including a depth camera and an infrared camera, etc., to ensure information security.

[0073] Palm image: It refers to the image taken for identifying the user's identity, which can be a 2D image taken by a 2D camera or a 3D image taken by a 3D camera.

[0074] Palm feature: It refers to the image feature obtained by extracting features from the palm image. It can be the palm image itself, such as the numerical information of each color channel of the image or the grayscale, brightness, etc. information of the image; or, it can also be the feature obtained by processing the palm image through image processing methods, such as performing binarization processing on the image; or, it can also be the image feature obtained by using an artificial neural network model learned based on deep learning methods to extract features from the palm image.

[0075] Identity recognition: It refers to the process of recognizing the corresponding identity information according to the biometric information of the target object. In the embodiments of the present application, it mainly involves identity recognition based on the method of collecting images of the target object, that is, identity recognition is achieved by identifying the biometric features of the target object from the collected images through image processing methods. For example, the biometric features can be palm features, pupil features, face features, etc.

[0076] Target object and detection object: In the embodiments of the present application, the target object refers to the party that triggers the rental operation, that is, the renter. In the actual scenario, when renting, it is often necessary to register the identity in the rental platform, so as to rent with the successfully registered identity, so that the rental platform can deduct fees, etc. for this identity after the rental process. Therefore, through the process of identity recognition, the identity of the target object in the rental platform can be recognized based on biometric information. Different from the target object, the detection object is from the perspective of the image, and refers to the object in the collected image, which can include the rental item and the renter in the image.

[0077] Item rental status: The item rental status is used to indicate whether the target object has successfully rented the rental item. Generally speaking, during the rental process, it is necessary to go through the process of unlocking the item and the target object taking out the rental item. Therefore, the item rental status can also include the status in different stages. For example, it can include the unlocking status of the rental item and the taking-out status of the target object taking out the rental item. Only when there is no abnormality in each stage, the item rental status can be normal accordingly, that is, the rental item is successfully rented.

[0078] Next, a brief description of the technical idea of the embodiments of the present application will be given.

[0079] In the related art, item rental has brought great convenience to daily life. For example, for the rental of power banks, due to hardware aging or problems with the device structure, there may be abnormal problems such as the failure to eject the power bank successfully, which greatly affects the normal use of the power bank by the renter. The same problem also exists in other rental scenarios, and the existence of these problems greatly affects the service quality of item rental. For example, for the rental of power banks, usually the battery power of the renter's terminal device (such as a mobile phone) is low. If a failure occurs, the process of reporting the fault, repairing, and then ejecting the power bank again will result in a long waiting time for the renter and a very poor experience.

[0080] Based on this, an embodiment of the present application provides a method for detecting the rental status of an item. In this method, when a target object triggers a rental operation, the FOV of the identity recognition device can be increased to collect a status detection image corresponding to the detection object. Then, based on the position change information of the detection object in the status detection image, the rental status of the item can be determined. It can be seen that this method utilizes the image acquisition ability of the identity recognition device. After triggering the rental operation, the FOV is further increased to better capture the detection object, and the position change information of the detection object is determined through image analysis, thereby determining the rental status of the item for the target object, such as whether the first rental item allocated to the target object is successfully unlocked and whether the target object successfully retrieves the first rental item. Thus, the detection of the rental status of the item is achieved without adding hardware conditions, improving the accuracy of perceiving the rental status of the item, so that relevant remedial measures or alarms can be taken in a timely manner when the rental status is abnormal, reducing the possibility that the lessee cannot use the rental item, and improving the service quality of item rental.

[0081] An embodiment of the present application detects the status at different stages through an image set to identify the rental status of the item at different stages, such as whether the first rental item allocated to the target object is successfully unlocked, and whether the target object successfully retrieves the first rental item after unlocking, thereby achieving comprehensive status detection and improving the accuracy of detecting the rental status of the item.

[0082] The following briefly introduces the application scenarios applicable to the technical solution of the embodiment of the present application. It should be noted that the application scenarios described below are only used to illustrate the embodiment of the present application rather than to limit it. In the specific implementation process, the technical solution provided by the embodiment of the present application can be flexibly applied according to actual needs.

[0083] The solution provided by the embodiment of the present application can be applied to any scenario involving item rental, such as power bank rental, stroller rental, bicycle rental, or 3D glasses rental scenarios. And the item rental involved in the embodiment of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, intelligent transportation, assisted driving, etc.

[0084] As Figure 1 shown, it is a schematic diagram of an application scenario provided by an embodiment of the present application. In this scenario, it may include item rental equipment 101, identity recognition server 102, and item rental server 103.

[0085] The item rental device 101 is a device for providing interaction to a target object at the front end, which may include an identity recognition device 1011 and an item management device 1012. Among them, the identity recognition device 1011 is used to collect the image of the target object to implement the identity recognition of the target object, and the item management device 1012 is used to control the unlocking of the rented item, etc.

[0086] Corresponding to different adopted identity recognition methods, the identity recognition device 1011 can be different devices. For example, when the palm brushing recognition method is adopted, the identity recognition device 1011 can be a palm brushing device, Figure 1 which is specifically shown by taking this as an example; or, when the face brushing recognition method is adopted, the identity recognition device 1011 can be a face brushing device; or, when the iris recognition method is adopted, the identity recognition device 1011 can be an iris recognition device. Of course, it can also be other devices based on the image recognition method, and this is not limited in the embodiments of the present application.

[0087] Similarly, corresponding to different rented items, the item management device 1012 can also be different devices. For example, when the rented item is a power bank, the item management device 1012 can be a power bank rental device; or, when the rented item is 3D glasses, the item management device 1012 can be a glasses rental device; or, when the rented item is a baby carriage, the item management device 1012 can be a baby carriage rental device. Of course, it can also be devices corresponding to any other possible rented items, and this is not limited in the embodiments of the present application.

[0088] The item rental device 101 can be installed with relevant applications (APPs), such as an identity recognition application and an item rental application. The applications involved in the embodiments of the present application can be software clients, or can also be clients such as web pages and mini programs, and the specific type of the client is not limited.

[0089] The identity recognition server 102 is the background server corresponding to the identity recognition application, which is used to implement identity recognition based on the image collected by the identity recognition device 1011, return the corresponding identity information to the item rental device 101, and can also perform rental fee payment, etc. based on the recognized identity.

[0090] The item rental server 103 is the background server corresponding to the item rental application, which is used to implement the management and maintenance of the rented items.

[0091] Among them, the identity recognition server 102 and the item rental server 103 can both be independent physical servers, or server clusters or distributed systems composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), as well as big data and artificial intelligence platforms, but are not limited thereto. And in some scenarios, the identity recognition server 102 and the item rental server 103 can be implemented by the same physical device, that is, the identity recognition server 102 and the item rental server 103 can be considered the same device.

[0092] In an actual scenario, taking the identity recognition device 1011 as a palm-sweeping device and the item management device 1012 as a power bank rental device as an example, when a target object needs to rent a power bank, a rental operation can be performed. Then, the palm-sweeping device will temporarily increase the FOV to collect relevant video streams including the power bank and the palm of the target object, and use this to detect whether the power bank pops out normally and whether the power bank is held in the retracting trajectory of the target object's hand, so as to determine whether the target object has taken the power bank normally, realize the detection of the rental status of the power bank, and when an abnormality occurs, such as the power bank fails to pop out normally, or the target object cannot take the power bank, relevant remedial measures can be taken in a timely manner to reduce the problem that the abnormal situation brings to the target object's inability to use normally and improve the service quality of item rental.

[0093] It should also be noted that since the process of collecting images of rental items and renters by the identity recognition device 1011 is involved in the embodiments of the present application, the identity recognition device 1011 needs to be able to collect rental items and renters. That is, in the specific implementation process, the orientation of the FOV of the identity recognition device 1011 may need to be configured according to requirements to meet the needs of image collection. Or, an identity recognition device 1011 that can achieve omnidirectional image collection can also be selected, or other image collection devices connected to the identity recognition device 1011 can be deployed at appropriate positions of the item rental device 101. The embodiments of the present application do not limit this.

[0094] It should be noted that the method for detecting the item rental status in the embodiments of the present application can be executed independently by any one of the item rental device 101, the identity recognition server 102, and the item rental server 103, or can be executed jointly by the above devices. For example, when executed independently by the item rental device 101, the item rental status detection process can be implemented independently by the item rental device 101. When jointly executed by the item rental device 101 and the identity recognition server 102, after the target trigger rental operation, the item rental device 101 can increase the FOV of the identity recognition device to collect a set of status detection images, and send the set of status detection images to the identity recognition server 102. Then, the identity recognition server 102 can use an image processing method to identify the position change information of the detection object, determine the final item rental status according to the position change information, and return the item rental status to the item rental device 101 to execute the subsequent control process. Of course, in actual applications, specific configurations can be made according to the situation, and the present application does not make specific limitations here.

[0095] Among them, the item rental device 101, the identity recognition server 102, and the item rental server 103 can all include one or more processors, memories, and an interactive I / O interface, etc. Among them, the memory can also store the program instructions required for each to execute the method for detecting the item rental status provided in the embodiments of the present application. When these program instructions are executed by the processor, they can be used to implement the item rental status detection process provided in the embodiments of the present application.

[0096] In the embodiments of the present application, the above-mentioned various devices can be directly or indirectly communicatively connected through one or more networks. The network can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network or a Wireless-Fidelity (WIFI) network. Of course, it can also be other possible networks, and the embodiments of the present application do not limit this. It should be noted that Figure 1 The above is only an example, and actually the number of terminal devices and servers is not limited, and no specific limitation is made in the embodiments of the present application.

[0097] See Figure 2 As shown, it is the system architecture provided by the embodiments of the present application, which is shown by taking palm brushing recognition as the identity recognition method as an example. In this architecture, the following two modules can be included:

[0098] (1) Item rental front end

[0099] The front end of item rental usually consists of two parts: an item rental application and an identity recognition application. The two can call each other during operation to cooperate in realizing the function of item rental. In some scenarios, the item rental application can be integrated into the identity recognition application in the form of a lightweight application. For example, if the identity recognition application supports the mini-program environment, the item rental application runs in this mini-program environment in the form of a mini-program. Figure 2 Taking this as a specific example for illustration, the front end of item rental can also be called the front end of identity recognition.

[0100] See Figure 2 As shown, the front end of item rental includes a 3D camera and a palm brushing application. The palm brushing application can call the 3D camera to collect images during operation to realize the function of the identity recognition device. Among them, the palm brushing application can include a palm brushing collection and recognition module, a mini-program module, an exception feedback module, and a status detection module.

[0101] The palm brushing collection and recognition module is used to collect the palm image of the target object and perform processing related to palm brushing recognition. For example, it can call the 3D camera to collect palm streaming media data, and optimize the streaming media data. The optimization is to select the optimal palm image through comprehensive evaluation of coefficient indicators such as palm size, angle, image contrast, image brightness, and clarity, and send it to the application back-end service for processing. When the conditions of the front-end device for item rental permit, this module can be used to perform image preprocessing and palm feature extraction on the palm image, which can be specifically configured according to the actual situation.

[0102] The mini-program module is used to provide a mini-program running environment.

[0103] The exception feedback module is used to give feedback when an exception occurs, such as giving feedback to the application back-end service or to the rental merchant service.

[0104] The status detection module is used to implement the item rental status detection function provided by the embodiments of the present application, including detecting the item unlocking status and the item taking-out status, etc., and can also be used for detecting other information.

[0105] (2) Application back-end service

[0106] The application back-end service is the back-end service corresponding to the palm brushing application. The application back-end service includes a payment service. The payment service includes two parts: a palm brushing recognition platform and a payment platform. The palm brushing recognition platform is used to realize functions related to palm brushing recognition. For example, it can identify the identity information of the target object according to the palm image sent by the front end. The payment platform realizes the payment process related to item rental. For example, after completing the item rental, it is necessary to pay the corresponding fee to the item rental provider. Then the payment platform can deduct the corresponding fee from the corresponding account of the identified identity information.

[0107] (3) Rental Merchant Services

[0108] The rental merchant service can be the backend service corresponding to the item rental application deployed in the front end, including a rental control platform for realizing functions such as scheduling, after-sales, and operation of rental items.

[0109] Next, in combination with the above-described application scenarios and system architectures, the method provided by the exemplary embodiments of the present application will be described with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown for the convenience of understanding the spirit and principle of the present application, and the embodiments of the present application are not limited in this regard.

[0110] See Figure 3 As shown, it is a flowchart of the item rental status detection method provided by the embodiments of the present application. This method can be executed by a computer device, and the computer device can be Figure 1 the item rental device or server shown. The specific implementation process of this method is as follows:

[0111] Step 301: In response to a rental operation triggered by a target object, adjust the FOV of the identity recognition device associated with the item rental device from a first FOV to a second FOV, where the identity recognition device is used for identity recognition based on the collected images, and the second FOV is greater than the first FOV.

[0112] In the embodiments of the present application, the target object refers to a user with an item rental demand, and the rental operation triggered by the target object can include any possible operations. For the convenience of understanding, some possible operations are exemplified here, that is, the rental operation can include but is not limited to any one of the following operations:

[0113] (1) The target object triggers an identity recognition operation. The identity recognition operation includes operations for identity recognition in the form of image acquisition, such as palm brushing operation, face brushing operation, or iris recognition operation. Taking the palm brushing operation as an example, the target object triggering the identity recognition operation can be that the target object places the palm in the image acquisition area of the palm brushing recognition device.

[0114] (2) The target object is within a preset distance range of the item rental device. That is, a preset distance range can be set in advance for the item rental device. When the target object is within this preset distance range, it indicates that the target object needs to rent an item, and the item rental process is started. A simple understanding is that when the target object approaches the item rental device, it is considered that the target object needs to rent an item. For example, the target object approaching the item rental device can mean that the distance between the target object and the item rental device is less than or equal to a certain distance threshold.

[0115] In the embodiments of the present application, the identity recognition device refers to a device that performs identity recognition based on the collected images. Generally speaking, in the conventional identity recognition process, when the FOV is too large, it is easy to collect interference information such as surrounding lights, which will interfere with the identity recognition. Therefore, the FOV is usually set to a smaller value, that is, the aforementioned first FOV. In the auxiliary recognition scenario, that is, when it is necessary to recognize the rental status of an item based on the images collected by the identity recognition device, since the possible positions of the detection object may not be on the main path of the camera, in order to improve the success rate of capturing the detection object, images should be collected in a larger range. Therefore, in the embodiments of the present application, after the target object triggers the rental operation, the FOV is adjusted from the first FOV to a second FOV that is greater than the first FOV. For example, in the scenario of power bank rental, it is usually necessary to detect whether the power bank is successfully unlocked (or ejected), and it is also necessary to detect whether the target object takes out the power bank after the power bank is unlocked. Therefore, in order to successfully capture the images of the power bank and the target object holding the power bank, the FOV of the identity recognition device can be set larger to improve the success rate of image capture.

[0116] Among them, the FOV in the embodiments of the present application can refer to the horizontal FOV, or can refer to the vertical FOV, or can refer to both the horizontal FOV and the vertical FOV, and can be specifically set according to the actual scenario. For example, it can be configured according to the installation position of the identity recognition device and the content of the images to be collected. Among them, the horizontal FOV refers to the FOV in the horizontal direction, and the horizontal direction can be defined as the direction parallel to the ground, and the vertical FOV refers to the FOV in the direction perpendicular to the ground.

[0117] In the embodiments of the present application, when the item rental device includes an identity recognition device, the identity recognition device associated with the item rental device can refer to the identity recognition device included in the item rental device itself; or, when the identity recognition device is an external device of the item rental device, that is, the item rental device and the identity recognition device are two independent devices, and the two are connected through a certain communication connection method, the identity recognition device associated with the item rental device can refer to the identity recognition device communicatively connected to the item rental device. Unless otherwise specified hereinafter, the description will be made taking the item rental device including the identity recognition device as an example.

[0118] It can be understood that adjusting the FOV of the identity recognition device specifically refers to adjusting the FOV of the camera included (or associated) with the identity recognition device. And in the present application, the specific values of the first FOV and the second FOV are not limited.

[0119] In the embodiments of the present application, the timing of adjusting the FOV includes but is not limited to the following timings:

[0120] Timing one: Adjust the FOV when the identity information of the target object is successfully recognized.

[0121] Specifically, when this method is adopted, after the target object triggers the rental operation, an identity recognition image of the target object can be collected at the first FOV in response to the rental operation, and the identity information of the target object can be recognized based on the identity recognition image. If the identity information of the target object is successfully recognized based on the identity recognition image, an adjustment instruction is sent to the identity recognition device, and the adjustment instruction is used to instruct the identity recognition device to adjust the field of view angle to the second FOV.

[0122] Among them, successfully recognizing the identity information of the target object can mean successfully determining the identity information of the target object from the identity information database based on the identity recognition image; or, it can also mean that after determining the identity information, it is determined that the identity information of the target object is a legal identity; or, it can also mean that after determining the identity information and the legitimacy verification of the identity information of the target object passes, it is further determined that the target object has the right to rent the item. The embodiments of the present application do not make specific limitations on this.

[0123] Timing two: Adjust the FOV when the first rental item is successfully allocated to the target object.

[0124] As described in the above "Timing one", after the target object confirms the identity information, the corresponding first rental item can be allocated to it based on this identity information. Thus, when the first rental item is successfully allocated to the target object, the adjustment of the FOV can be triggered. For example, an adjustment instruction can be sent to the identity recognition device, and the adjustment instruction is used to instruct the identity recognition device to adjust the field of view angle to the second field of view angle.

[0125] In a possible implementation manner, after the target object confirms the identity information, the rental service provider applet can select the first rental item allocated to the target object this time from the rental items associated with the item rental device. For example, the rental service provider applet can maintain the status of each rental item, such as the idle status and the item attribute status, and then allocate the first rental item to the target object according to the status.

[0126] Among them, the idle status indicates whether the rental item has been rented, that is, whether it is currently in an idle state. The item attribute status is used to indicate the available degree of the rental item. The available degree can be related to the type of the item. When the types of the items are different, corresponding available degree parameters can be configured. For example, for a power bank, the item attribute status can be the remaining power of the power bank, and when allocating, the power bank with a higher remaining power can be preferentially allocated. Or for a bicycle, the item attribute status can be the status of each component, and when allocating, the bicycle with all components in good condition can be preferentially allocated. The selection of the parameters of the item attribute status will not be listed one by one here and can be configured according to the actual situation.

[0127] In a possible implementation, after the target object confirms the identity information, the background of the rental service provider's mini-program can select the first rental item allocated to the target object this time from the rental items associated with the item rental device. Similarly, the rental service provider's mini-program can synchronize the status of each rental item to the background, and then the background can allocate the first rental item to the target object according to the status. After the background selects the first rental item allocated to the target object this time, it can send the item information of the first rental item, such as the item ID, to the rental service provider's mini-program.

[0128] Furthermore, the rental service provider's mini-program can transmit the item information of the first rental item to the palm-sweeping application through the mini-program running framework. Then, when the palm-sweeping application learns that the first rental item has been allocated to the target object, it will perform the adjustment of the FOV.

[0129] Among them, the item information received by the rental service provider's mini-program from the background may include the item ID, and the item information transmitted by the rental service provider's mini-program to the palm-sweeping application may include the position information corresponding to the first rental item, so as to facilitate the palm-sweeping application to locate the position of the first rental item in the image. Of course, in addition to the position information, the item information transmitted by the rental service provider's mini-program to the palm-sweeping application may also include information such as the item ID.

[0130] For example, taking a power bank as an example, after the background of the rental service provider's mini-program confirms the power bank ID that should be popped up this time, it will send the ID to the rental service provider's mini-program, and the rental service provider's mini-program will then send the position information corresponding to the ID to the palm-sweeping application, so that the palm-sweeping application performs the adjustment of the FOV.

[0131] Timing three: Adjust the FOV when the target object triggers the rental operation.

[0132] In this way, once the target object triggers the rental operation, such as approaching the item rental device or performing an identity recognition operation, the FOV is immediately adjusted. For example, an adjustment instruction can be sent to the identity recognition device, and the adjustment instruction is used to instruct the identity recognition device to adjust the field of view angle to the second field of view angle.

[0133] Of course, in addition to the above-mentioned "Timing One" to "Timing Three", the FOV adjustment can also be triggered at other times, and the embodiments of the present application do not limit this.

[0134] Exemplarily, taking the power bank rental as an example, see Figure 4 As shown, it is a schematic diagram for adjusting the FOV. Among them, the camera direction of the palm-sweeping recognition device can face the direction of the power bank control device. For example, it can be vertically upward from the ground. In this way, not only can the image acquisition requirements for palm-sweeping recognition be met, but also the image acquisition requirements for detecting the rental status of the power bank can be met. Furthermore, when the above-mentioned timing is satisfied, the FOV adjustment is performed. FromFigure 4 The first FOV shown is adjusted to a second FOV, thereby increasing the range of the image acquisition area, avoiding missing information acquisition, and improving the accuracy of status detection.

[0135] Step 302: After the identity recognition device recognizes the identity information of the target object, obtain a set of status detection images acquired at the second field of view angle. The status detection images included in the set of status detection images present at least one detection object, and the at least one detection object includes a first rental item assigned to the target object.

[0136] In the embodiments of the present application, usually after a first rental item is assigned to the target object, it is necessary to detect the rental status of the item. If the target object fails to rent the item, in essence, there is no need to detect the rental status of the item. Therefore, the acquisition of status detection images can be performed after the identity information of the target object is recognized.

[0137] To avoid missing information acquisition, after adjusting the FOV, that is, the second FOV can be continuously used for image acquisition to obtain an image set after adjusting the FOV. Furthermore, when status detection is required, obtain the image set acquired by the identity recognition device after recognizing the identity information of the target object from the image set, and use the obtained image set as the set of status detection images. Of course, it is also possible to perform image acquisition at the second FOV only after recognizing the identity information of the target object to obtain the set of status detection images, thereby avoiding occupying too much storage resources.

[0138] Among them, the above-mentioned after recognizing the identity information of the target object may refer to after successfully recognizing the identity information of the target object, or may refer to recognizing the identity information of the target object and after assigning a first rental item to the target object.

[0139] To facilitate the description of the identity recognition process, here, taking the identity recognition device as a palm-sweeping device and the identity recognition image as a palm image as an example for introduction, for other identity recognition methods, the following description can also be referred to.

[0140] See Figure 5 shown, which is a schematic flowchart of palm-sweeping to recognize identity information. Here, specifically taking a small program accessing an item rental application as an example, as Figure 5 shown, in the item rental device, starting from the palm-sweeping application, the palm-sweeping application provides the core palm-sweeping recognition ability and payment service. After the target object sweeps their palm on the item rental device, the palm-sweeping application can further start the small program framework to provide a small program running environment, such as the WeChat Mini-Program Framework (WMPF), and then start the rental service provider's small program in the small program running environment.

[0141] When performing identity recognition, the palm-sweeping application can collect the palm streaming media data of the target object by calling the 3D camera integrated in the item rental device. After obtaining the palm streaming media data, the palm-sweeping application can optimize the palm streaming media data. The optimization is to select the optimal palm image through comprehensive evaluation of coefficient indicators such as palm size, angle, image contrast, image brightness, and clarity, and send it to the backend service for identity recognition. After the backend receives the palm data uploaded by the palm-sweeping application and extracts features, it retrieves the identity information of the target object from the entire database and returns the identity information to the palm-sweeping application. Then, the palm-sweeping application passes the obtained identity information of the target object to the rental service provider's applet through the applet operation framework. Furthermore, the rental service provider's applet can obtain the identity information of the target object and perform corresponding processing. For example, it can display the identity information through the applet interface and prompt the target object to confirm. After the target object confirms that the identity information is correct, the first rental item can be allocated to it. For example, when the rental item is a power bank, the power bank applet can allocate the corresponding power bank to the target object after the target object confirms its identity, and control the item rental device to eject the power bank so that the target object can take it out.

[0142] In the embodiments of the present application, the identity information can be any information used to distinguish an object, such as the object's ID number, mobile phone number, or the identity identifier of the object in the palm-sweeping application, for example, it can be an account identifier (such as openid).

[0143] Specifically, when performing identity recognition, feature extraction can be performed on the palm image to obtain the target palm features of the target object, and the target palm features are matched with at least one reference palm feature in the database to determine the reference palm feature that matches the target palm feature. If there is a reference palm feature that matches the target palm feature, it indicates that the target object has registered its identity, and the identity information associated with the reference palm feature can be determined as the identity information of the target object.

[0144] It should be noted that, when the conditions of the item rental device permit, the item rental device itself can also perform the identity recognition process.

[0145] In the embodiments of the present application, the state detection image set includes state detection images in which at least one detection object appears, which means that at least one detection object is included in one or more state detection images. The at least one detection object includes the first rental item allocated to the target object. In addition to the first rental item, the detection object can also include the limbs of the target object. The type of the limbs of the target object can be related to the type of the rental item. For example, when the rental item is a power bank or 3D glasses, usually the target object will hold the power bank or 3D glasses to take them out. Therefore, the limb can refer to the hand of the target object.

[0146] Step 303: Determine the position change information of each of at least one detection object between each state detection image in the state detection image set according to the state detection image set.

[0147] In the embodiments of the present application, when the state of the leased item changes, the image content presented will also change. For example, the position where the leased item is located will change. Therefore, the position change information can be detected through images to determine whether the state of the leased item is normal. Taking a power bank as an example, when the power bank pops out successfully, there should be a difference in the position of the power bank before and after it pops out, and this difference is predictable. Therefore, it is possible to combine the preset position change information to determine whether the power bank pops out normally.

[0148] In a possible implementation manner, the position change information may include the following information:

[0149] (1) The first actual movement trajectory of the first leased item. That is, the movement trajectory of the first leased item can be used as a measure of the position change information of the first leased item. Correspondingly, the preset movement trajectory of the leased item can be pre-configured for subsequent state recognition.

[0150] See Figure 6 As shown, for the process of item leasing, after the target object confirms the lease, it usually needs to go through the leased item unlocking stage and the leased item taking-out stage. Among them, in the leased item unlocking stage, it is necessary to perform an unlocking operation on the first leased item allocated to the target object so that the target object can take out the first leased item. In this stage, it is necessary to detect whether the first leased item is successfully unlocked. Taking the power bank lease as an example, the power bank usually needs to pop out (i.e., unlock) first before the user can take it away. If the power bank fails to pop out due to equipment hardware reasons, the user cannot take it away. Therefore, in the leased item unlocking stage, the detection target is to detect whether the power bank pops out normally. See Figure 7 As shown, taking the power bank as an example, from Figure 7 it can be seen that in the original state, the states of each power bank are the same. However, during the pop-out process and in the successfully popped-out state, the first leased item (i.e., the power bank in the upper right corner) allocated to the target object is different from other power banks in the image, and compared with the original state, its position occupied in the image also changes. Therefore, the actual movement trajectory of the power bank can be determined according to the collected image sequence to assist in judging whether it pops out successfully.

[0151] In the leased item taking-out stage, the target object will take out the first leased item. In this stage, it is necessary to detect whether the first leased item is successfully taken out. Similarly, taking the power bank lease as an example, after the power bank pops out, the user usually holds the power bank and takes it out. The detection target in this stage is to detect whether the power bank is taken away normally. SeeFigure 7 As shown, after the power bank successfully pops out, the user can reach out to take out the power bank. The taking-out process may include the reaching-out process and the retracting process. During the retracting process, the user needs to hold the power bank. Therefore, it is possible to identify whether the power bank is held during the retracting process based on the collected image sequence, or the actual movement trajectory of the power bank can also be determined to assist in judging whether it is successfully taken out.

[0152] (2) The second actual movement trajectory of the limb of the target object. As described above, the movement trajectory of the limb is mainly used to assist in detecting whether the item is successfully taken out during the taking-out stage of the rental item.

[0153] (3) The pixel feature change information of the position where the first rental item is located, which is used to indicate whether there is a change in this position to reflect whether the first rental item is successfully unlocked, etc.

[0154] Of course, in addition to the above information, other information can also be used as the position change information, and the embodiments of the present application do not limit this.

[0155] Next, the first actual movement trajectory of the first rental item will be described first.

[0156] Specifically, the above state detection image set may include a first image subset, and the first image subset includes a reference image and an image sequence located after the image acquisition moment of the reference image. For example, the reference image may be the first acquired image, or an image of the first rental item in its original state. For example Figure 7 the original state image of the power bank shown in

[0157] In a possible implementation manner, when determining the first actual movement trajectory, the original position of the first rental item in the reference image can be determined according to the first item information corresponding to the first rental item, such as the position information or number of the first rental item in the item rental device, etc., which can distinguish different rental items. For example, taking Figure 7 the power bank image shown in as an example, when the first rental item is the power bank in the upper right corner, the corresponding position of the power bank in the image can be determined according to its position information in the item rental device. This position may be, for example, an image area determined with the upper left corner of the area where the power bank is located as the starting point and constrained by the length and width segments.

[0158] Furthermore, according to the first item information and the original position, the actual positions of the first rental item in each image in the image sequence can be determined to form a position sequence of the first rental item in the image sequence. Finally, based on the original position and the position sequence, the first actual movement trajectory of the first rental item can be determined.

[0159] In the embodiments of the present application, since there is continuity between the unlocking stage and the taking-out stage, it is possible to first determine whether the first rental item is successfully unlocked. In other words, the function of the first image subset collected at this stage is to determine whether the first rental item is successfully unlocked.

[0160] See Figure 8A As shown, taking the rental of a power bank as an example, according to the status detection image collected during the ejection process, the actual position of the power bank in the image can be determined. Furthermore, by combining the original state of the power bank with its actual position during the process from the original state to the successful ejection state, the actual movement trajectory of the power bank can be determined. Similarly, in the subsequent taking-out stage, this method can also be used to determine the actual movement trajectory of the power bank.

[0161] It should be noted that the first actual movement trajectory of the first rental item may include trajectory 1 during the unlocking stage (as Figure 8A shown) and trajectory 2 during the taking-out stage. The determination methods of trajectory 2 and the second actual movement trajectory of the limb of the target object can also adopt the above similar methods, so they will not be elaborated here.

[0162] In a possible implementation manner, considering that after the rental item is successfully unlocked, the pixel features at the position where the rental item is located will change, so it is possible to determine whether the first rental item is successfully unlocked according to the pixel features.

[0163] Then, when determining the first actual movement trajectory, the original position of the first rental item in the reference image can also be determined according to the first item information corresponding to the first rental item, and the reference pixel features corresponding to the original position can be extracted from the reference image. For each image in the image sequence, the target pixel features corresponding to the original position are respectively extracted. Furthermore, according to the feature differences between the reference pixel features and the obtained target pixel features, the position change information of the first rental item can be determined. This position change information can characterize whether the pixel features at the position where the first rental item is located have changed, so as to determine the unlocking state of the first rental item.

[0164] Step 304: Determine the item rental status of the target object for the first rental item according to the position change information corresponding to each of at least one detection object.

[0165] As shown above, the item rental status may include the unlocking state and the taking-out state of the first rental item. Next, the determination of the unlocking state of the first rental item will be introduced first.

[0166] In a possible implementation manner, see Figure 6As shown, when the position change information includes the first actual movement trajectory of the first leased item, when determining the unlocking state of the first leased item, the first actual movement trajectory can be matched with a preset first reference movement trajectory. If the first actual movement trajectory matches the preset first reference movement trajectory, it is determined that the unlocking state of the first leased item is successful. Otherwise, if the first actual movement trajectory does not match the first reference movement trajectory, it is determined that the unlocking state of the first leased item is failed.

[0167] Among them, the first reference movement trajectory is the movement trajectory of a leased item that can be normally unlocked, which can be configured according to experience or determined according to the actual unlocking process of a normally unlocked leased item.

[0168] It should be noted that the matching of the first actual movement trajectory and the first reference movement trajectory means that the trajectory directions of the two are the same, or the difference between the two is within an allowable error range.

[0169] In a possible implementation manner, when the position change information includes the pixel feature change of the position where the first leased item is located, when determining the unlocking state of the first leased item, the actually collected pixel feature change can be matched with the reference pixel feature change. If they match, it is determined that the unlocking state of the first leased item is successful. Otherwise, if they do not match, it is determined that the unlocking state of the first leased item is failed.

[0170] Taking a power bank as an example, when using this method, it can be determined whether the color of the position corresponding to the specified power bank in the image changes as expected. If it is satisfied, it means that it pops up normally. If the color does not change, it means that the power bank fails to pop up normally. Of course, in addition to color, the pixel feature can also include other possible feature information, or the pixel feature can be a feature vector obtained by feature extraction using a feature extraction model based on deep learning. The embodiments of the present application do not limit this.

[0171] In the embodiments of the present application, if the unlocking of the first leased item fails, the target object cannot take out the first leased item, then it can be determined that the item rental state is abnormal, and then the subsequent fault repair process can be entered. If the unlocking of the first leased item is successful, then refer to Figure 6 As shown, the removal state of the first leased item can be continuously detected. The judgment of the removal state will be introduced below.

[0172] In a possible implementation manner, the removal state of the first leased item can be judged according to the trajectory of the limb and the first leased item.

[0173] Specifically, considering that if the first rental item is successfully taken out, the movement trajectories of the limb and the first rental item should be the same, and their positional relationship is relatively fixed during the movement. Therefore, preset conditions for the movement trajectory can be preconfigured. When the preset conditions are met, it is determined that the first rental item is successfully taken out; if the preset conditions are not met, it is determined that the first rental item fails to be successfully taken out.

[0174] Among them, the preset conditions may include: the second actual movement trajectory matches the first actual movement trajectory, and the distances between the limb and the first rental item in each image corresponding to the second actual movement trajectory are not greater than the preset threshold. It should be noted that the first actual movement trajectory here should be understood as the above-mentioned trajectory 2.

[0175] Therefore, if the second actual movement trajectory meets the preset conditions, it can be determined that the take-out status of the first rental item is successful; otherwise, if the second actual movement trajectory does not meet the preset conditions, that is, the second actual movement trajectory does not match the first actual movement trajectory, or the distances between the limb and the first rental item in each image corresponding to the second actual movement trajectory are greater than the preset threshold, it indicates that the first rental item does not move following the limb, and it is determined that the unlocking status of the first rental item is a failed take-out.

[0176] Taking the rental of a power bank as an example, see Figure 8B As shown, during the power bank take-out stage, the movement trajectory of the power bank is trajectory 2, and the movement trajectory of the hand is trajectory 3 (here it refers to the hand-retracting trajectory. In actual application, when the image corresponding to the hand-retracting trajectory is recognized according to image recognition, the subsequent image recognition trajectories 3 and 2 can be used). If trajectory 2 and trajectory 3 match, and the distance between the power bank and the hand is less than the preset threshold during this process, it indicates that the power bank moves following the hand, which means the target object has taken out the power bank. On the contrary, when these conditions are not met, it indicates that the target object has not taken out the power bank.

[0177] In a possible implementation manner, an image recognition method can also be used to determine the take-out status of the first rental item.

[0178] Specifically, the state detection image set includes a second image subset, and the second image subset is located after the first image subset in terms of the acquisition time. Alternatively, it can be determined according to whether there is a user's limb in the image recognition screen and the movement direction of the limb. When the movement direction of the limb suddenly changes (or reverses), the image collected after this image can be used as the second image subset.

[0179] In the embodiments of the present application, an image recognition model can be pre-trained to identify the interaction type between the limb and the first rental item in the image. Among them, the interaction type can be configured according to the actual scenario. For example, for the rental of a power bank or 3D glasses, the interaction type can be holding the power bank or 3D glasses. When training the model, multiple training samples can be collected in advance. Each training sample includes a sample image and its corresponding training label, and the training label is used to indicate the interaction type between the limb and the rental item in the corresponding image. Then, after training, the image recognition model can be used to identify the interaction type. Among them, the image recognition model can be implemented by an item recognition model.

[0180] Furthermore, each image included in the second image subset can be input into the trained image recognition model to obtain the corresponding image recognition result for each image. Each image recognition result is used to indicate the interaction type between the limb and the first rental item in the corresponding image. Then, according to the obtained image recognition results, the removal state can be determined. For example, when the image recognition results corresponding to most images all indicate that the interaction type is the specified interaction type, it is determined that the removal state is a successful removal; otherwise, it is determined that the removal state is a failed removal.

[0181] Taking the rental of a power bank as an example, after the target object takes out the power bank, it is necessary to ensure that the power bank device is included in the retracting trajectory of the target object's hand. Therefore, this purpose can be achieved through item recognition. Specifically, it is necessary to collect an image dataset containing a hand-held power bank in advance and preprocess the image data, including noise reduction, resizing, and grayscaling, etc. Then, a deep learning model, such as a convolutional neural network (CNN), is used to extract features from the image data, convert the image into a vector representation, and use the labeled image dataset to train the model so that it can recognize a hand-held power bank. In addition, another set of image datasets can be used to evaluate the model, and metrics such as the accuracy, recall rate, and F1 score of the model are calculated. When meeting the usage requirements, the model is put into actual detection to detect whether the interaction type of the actually collected image is a hand-held power bank, assisting in the detection of the removal state.

[0182] In a possible implementation manner, an image segmentation method can also be used to determine the removal state of the first rental item.

[0183] Specifically, for each state detection image included in the second image subset, each state detection image is respectively input into the trained image segmentation model to obtain the image segmentation result corresponding to the state detection image. The image segmentation result includes the position information of the limbs of the target object and the first rental item in a state detection image, that is, the areas corresponding to the limbs and the first rental item in the image are segmented out, and the corresponding position information of each is obtained, so as to determine the relative position information between the limbs and the first rental item according to the position information of the limbs and the first rental item respectively. Furthermore, the taking-out state can be determined according to the relative position information corresponding to each state detection image and the preset position constraint conditions.

[0184] Among them, the position constraint condition can be, for example, that the relative position information remains unchanged, or the difference between each relative position information is within the set allowable range. When the relative position information corresponding to each state detection image meets the position constraint condition, it is determined that the taking-out state is successful; on the contrary, it can be determined that the taking-out state is failed. Of course, in actual scenarios, other position constraint conditions can also be set according to the requirements of different scenarios, and the embodiments of the present application do not limit this.

[0185] In the embodiments of the present application, when the item rental state is unlocking failure or taking-out failure, it indicates that the item rental state is abnormal and corresponding fault repair is required. Therefore, at least one of the following operations can be performed:

[0186] (1) Reassign a second rental item for the target object from the rental items in the idle state, that is, reassign other rental items for the target object to ensure that the target object can use the rental service normally and improve the service quality.

[0187] (2) Generate an alarm message based on the item rental state and send the alarm message to the background server so that relevant personnel can find and repair the fault in time.

[0188] Of course, other possible operation methods may also be included, and the embodiments of the present application do not limit this.

[0189] Next, taking the power bank rental as an example, the solution of the embodiments of the present application will be introduced. When the target object performs a rental operation, the identity recognition device uploads the optimal palm image collected to the backend service. After the backend service receives the optimal palm image uploaded by the identity recognition device and extracts the features, it retrieves and identifies the identity information of the target object in the entire database, such as openID. At the same time, in order to reduce the problem of high time consumption brought by the small program, the weakening of the small program login state can be realized in combination with openID, that is, using openID as the only KEY to associate with the identity information of the object.

[0190] SeeFigure 9 As shown, in order to improve the success rate of capturing the trajectory of the handheld power bank, the FOV of the identity recognition device is also adjusted, changing the FOV from the first FOV to the second FOV. Then, images are collected in the FOV acquisition state, and the position change information of the detection object is identified based on the state detection image set. The rental status (or power bank release status) of the target object with respect to the power bank is determined according to the position change information.

[0191] For example Figure 9 As shown, it may include whether the unlocking is successful and the successful removal. That is, the process of detecting based on the state detection image set may include the following judgment of trajectory information:

[0192] Trajectory 1: According to the position of the power bank corresponding in the image, for example, with the upper left corner as the coordinate, a position can be determined by adding length and width constraints, and then it is judged whether the color of this position changes as required. If the requirement is met, it indicates that the power bank pops out normally.

[0193] Trajectory 2: After the power bank pops out normally through image recognition, it is identified whether the power bank device is included in the trajectory of the target object taking the power bank after taking it out. If it is included, it indicates that the target object successfully takes out the power bank.

[0194] When both the unlocking and the removal status are successful, it indicates that the target object successfully takes away the power bank. On the contrary, when the unlocking fails or the removal fails, a new power bank needs to be allocated to this target object, and an alarm message is uploaded to the background server.

[0195] In summary, in the embodiment of the present application, the detection of the rental status of the rental item is realized by combining the camera for identity recognition, reducing the impact of abnormal status on the user experience. For example, in combination with the palm - swiping camera, after the user swipes the palm, the FOV is further enlarged, and image data is continuously collected to judge whether the power bank pops out and whether the power bank moves along the established trajectory after the user reaches out. If the requirements are not met, an alarm will be generated, a new power bank will be popped out again, and the abnormal status will be uploaded to the cloud to avoid affecting the user experience and perform parallel maintenance.

[0196] Please refer to Figure 10 , based on the same inventive concept, the embodiment of the present application also provides an item rental status detection device 100, and this device includes:

[0197] An adjustment unit 1001, configured to adjust the field of view angle of the identity recognition device associated with the item rental device from a first field of view angle to a second field of view angle in response to a rental operation triggered by a target object. The identity recognition device is used for identity recognition according to the collected images, and the second field of view angle is greater than the first field of view angle;

[0198] An image acquisition unit 1002, configured to obtain a state detection image set acquired at the second field of view angle after the identity recognition device recognizes the identity information of the target object, where at least one detection object is presented in the state detection images included in the state detection image set, and the at least one detection object includes a first rental item assigned to the target object;

[0199] An image processing unit 1003, configured to determine, according to the state detection image set, position change information of each of the at least one detection object between the state detection images in the state detection image set;

[0200] A state determination unit 1004, configured to determine an item rental state of the target object for the first rental item according to the position change information corresponding to each of the at least one detection object.

[0201] In a possible implementation manner, the position change information includes a first actual movement trajectory of the first rental item, and the item rental state includes an unlocking state of the first rental item; then the state determination unit 1004 is specifically configured to:

[0202] If the first actual movement trajectory matches a preset first reference movement trajectory, determine that the unlocking state of the first rental item is successful unlocking;

[0203] If the first actual movement trajectory does not match the first reference movement trajectory, determine that the unlocking state of the first rental item is failed unlocking.

[0204] In a possible implementation manner, the state detection image set includes a first image subset, and the first image subset includes a reference image and an image sequence after the image acquisition moment and located after the reference image; then the image processing unit 1003 is specifically configured to:

[0205] Determine an original position of the first rental item in the reference image according to first item information corresponding to the first rental item;

[0206] Determine a position sequence of the first rental item in the image sequence according to the first item information and the original position, where the position sequence includes actual positions of the first rental item in each image in the image sequence;

[0207] Determine the first actual movement trajectory according to the original position and the position sequence.

[0208] In a possible implementation manner, the item rental state includes an unlocking state of the first rental item, the state detection image set includes a first image subset, and the first image subset includes a reference image and an image sequence after the image acquisition moment and located after the reference image; then the state determination unit 1004 is specifically configured to:

[0209] Determine the original position of the first leased item in the reference image according to the first item information corresponding to the first leased item;

[0210] Extract the reference pixel features corresponding to the original position from the reference image;

[0211] Extract the target pixel features corresponding to the original position from each image in the image sequence respectively;

[0212] Determine the unlocking state according to the feature differences between the reference pixel features and the obtained target pixel features.

[0213] In a possible implementation manner, the position change information further includes the second actual movement trajectory of the limb of the target object, and the item rental state includes the taking-out state of the first leased item; then the state determination unit 1004 is further specifically configured to:

[0214] If the second actual movement trajectory meets the preset conditions, determine that the taking-out state of the first leased item is successful;

[0215] If the second actual movement trajectory does not meet the preset conditions, determine that the unlocking state of the first leased item is a failure to take out;

[0216] Wherein, the preset conditions include: the second actual movement trajectory matches the first actual movement trajectory, and the distance between the limb and the first leased item in each image corresponding to the second actual movement trajectory is not greater than the preset threshold.

[0217] In a possible implementation manner, the state detection image set includes a second image subset, and the item rental state includes the taking-out state of the first leased item; then the state determination unit 1004 is further specifically configured to:

[0218] Input each image included in the second image subset into the trained image recognition model to obtain the image recognition result corresponding to each image, and the image recognition result is used to indicate the interaction type between the limb and the first leased item in the corresponding image;

[0219] Determine the taking-out state according to the obtained image recognition results of each image;

[0220] Wherein, the state detection model is trained based on a plurality of training samples, and each training sample in the plurality of training samples includes a sample image and its corresponding training label, and the training label is used to indicate the interaction type between the limb and the leased item.

[0221] In a possible implementation manner, the state detection image set includes a second image subset, and the item rental state includes the taking-out state of the first leased item; then the state determination unit 1004 is further specifically configured to:

[0222] For each state detection image included in the second image subset, the following operations are respectively performed:

[0223] For a state detection image, input the state detection image into the trained image segmentation model to obtain an image segmentation result corresponding to the state detection image. The image segmentation result includes the position information of the limbs of the target object and the first rental item in a state detection image;

[0224] According to the position information of the limbs and the first rental item respectively, determine the relative position information between the limbs and the first rental item;

[0225] According to the relative position information corresponding to each state detection image and the preset position constraint conditions, determine the removal state.

[0226] In a possible implementation manner, the device further includes an execution unit 1005, configured to:

[0227] If the item rental state is unlocking failure or removal failure, perform at least one of the following operations:

[0228] Reassign a second rental item for the target object from the rental items in the idle state;

[0229] Generate an alarm message based on the item rental state and send the alarm message to the background server.

[0230] In a possible implementation manner, the adjustment unit 1001 is specifically configured to:

[0231] In response to the rental operation, collect an identity recognition image of the target object at the first field of view angle;

[0232] If the identity information of the target object is successfully recognized according to the identity recognition image, send an adjustment instruction to the identity recognition device. The adjustment instruction is used to instruct the identity recognition device to adjust the field of view angle to the second field of view angle.

[0233] In a possible implementation manner, the identity recognition device is a palm brushing device, and the identity recognition image is a palm image; then the image processing unit 1003 is further configured to:

[0234] Extract features from the palm image to obtain the target palm features of the target object;

[0235] Match the target palm features with at least one reference palm feature to determine the reference palm feature that matches the target palm features;

[0236] Determine the identity information associated with the reference palm feature as the identity information of the target object.

[0237] Through the above device, by means of the image acquisition ability of the identity recognition device, after triggering the rental operation, the FOV is further increased to better capture the detection object, and the position change information of the detection object is determined through image analysis, so as to determine the item rental status of the target object for the rented item. For example, whether the first rented item assigned to the target object is successfully unlocked and whether the target object successfully takes out the first rented item, etc., so as to detect the item rental status without increasing hardware conditions, improve the accuracy of item rental status perception, so that relevant remedial measures or alarms can be taken in time when the rental status is abnormal, reduce the possibility that the lessee cannot use the rented item, and improve the service quality of item rental.

[0238] This device can be used to execute the methods shown in the embodiments of the present application. Therefore, for the functions that can be realized by each functional module of this device, reference can be made to the descriptions of the foregoing embodiments, and details will not be repeated.

[0239] Please refer to Figure 11 , based on the same technical concept, the embodiments of the present application also provide a computer device. In one embodiment, the computer device can be, for example, Figure 1 the identity recognition server or the item rental server shown, and the computer device is as Figure 11 shown, including a memory 1101, a communication module 1103, and one or more processors 1102.

[0240] The memory 1101 is used to store the computer program executed by the processor 1102. The memory 1101 mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system and programs required to run the functions of the embodiments of the present application; the data storage area can store various function information and operation instruction sets, etc.

[0241] The memory 1101 can be a volatile memory, such as a random-access memory (RAM); the memory 1101 can also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); or the memory 1101 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1101 can be a combination of the above memories.

[0242] The processor 1102 may include one or more central processing units (CPUs) or be a digital processing unit, etc. The processor 1102 is used to implement the above-mentioned item rental status detection method when calling the computer program stored in the memory 1101.

[0243] The communication module 1103 is used to communicate with the terminal device and other servers.

[0244] In the embodiments of the present application, the specific connection medium between the above-mentioned memory 1101, communication module 1103, and processor 1102 is not limited. In the embodiments of the present application Figure 11 it is shown that the memory 1101 and the processor 1102 are connected through a bus 1104. The bus 1104 is described in thick lines in Figure 11 The connection manners between other components are only for illustrative purposes and are not to be taken as limiting. The bus 1104 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of description, Figure 11 it is only described by a thick line in

[0245] The memory 1101 stores a computer storage medium, and the computer storage medium stores computer-executable instructions. The computer-executable instructions are used to implement the item rental status detection method of the embodiments of the present application, and the processor 1102 is used to execute the item rental status detection method of the above-mentioned various embodiments.

[0246] In another embodiment, the computer device may also be a terminal device, such as Figure 1 the item rental device shown. In this embodiment, the structure of the computer device may be as shown in Figure 12 and includes components such as a communication component 1210, a memory 1220, a display unit 1230, a camera 1240, a sensor 1250, an audio circuit 1260, a Bluetooth module 1270, and a processor 1280.

[0247] The communication component 1210 is used to communicate with the server. In some embodiments, it may include a Wireless Fidelity (WiFi) module. The WiFi module belongs to short-range wireless transmission technology, and the computer device can help send and receive information through the WiFi module.

[0248] The memory 1220 can be used to store software programs and data. The processor 1280 executes various functions and data processing of the terminal device by running the software programs or data stored in the memory 1220. The memory 1220 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices. The memory 1220 stores an operating system that enables the terminal device to run. In this application, the memory 1220 can store the operating system and various application programs, and can also store the code for implementing the article rental status detection method of the embodiments of this application.

[0249] The display unit 1230 can also be used to display the information input by the user or the information provided to the user, as well as the graphical user interface (GUI) of various menus of the terminal device. Specifically, the display unit 1230 may include a display screen 1232 disposed on the front of the terminal device. Among them, the display screen 1232 can be configured in the form of a liquid crystal display, a light-emitting diode, etc. The display unit 1230 can be used to display the article rental-related pages in the embodiments of this application.

[0250] The display unit 1230 can also be used to receive input numerical or character information and generate signal inputs related to the user settings and function control of the terminal device. Specifically, the display unit 1230 may include a touch screen 1231 disposed on the front of the terminal device, which can collect touch operations of the user thereon or nearby, such as clicking buttons, dragging scroll boxes, etc.

[0251] Among them, the touch screen 1231 can cover the display screen 1232, or the touch screen 1231 and the display screen 1232 can be integrated to implement the input and output functions of the terminal device. After integration, it can be simply called a touch display screen. In this application, the display unit 1230 can display application programs and corresponding operation steps.

[0252] The camera 1240 can be used to capture static images. The user can post comments on the images captured by the camera 1240 through an application. The camera 1240 can be one or multiple. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the processor 1280 to convert it into a digital image signal. For example, it may include the aforementioned 3D camera for implementing identity recognition processes such as palm brushing recognition, and for collecting the status detection images involved in the embodiments of this application.

[0253] The terminal device may further include at least one sensor 1250, such as an acceleration sensor 1251, a distance sensor 1252, a fingerprint sensor 1253, and a temperature sensor 1254. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0254] The Bluetooth module 1270 is used to interact with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the terminal device can establish a Bluetooth connection with a wearable computer device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 1270, so as to perform data interaction.

[0255] The processor 1280 is the control center of the terminal device, connecting various parts of the entire terminal using various interfaces and lines. By running or executing software programs stored in the memory 1220, and by calling data stored in the memory 1220, it executes various functions of the terminal device and processes data. In some embodiments, the processor 1280 may include one or more processing units; the processor 1280 may also integrate an application processor and a baseband processor. Among them, the application processor mainly processes the operating system, user interface, and application programs, etc., and the baseband processor mainly processes wireless communication. It can be understood that the above baseband processor may not be integrated into the processor 1280. In this application, the processor 1280 can run the operating system, application programs, user interface display, and touch response, as well as the item rental status detection method of the embodiments of this application. In addition, the processor 1280 is coupled to the display unit 1230.

[0256] Based on the same inventive concept, an embodiment of this application also provides a computer storage medium. This computer storage medium stores a computer program. When this computer program runs on a computer device, it causes the computer device to execute the steps in the item rental status detection method according to various exemplary embodiments of this application described above in this specification.

[0257] In some possible implementation manners, each aspect of the item rental status detection method provided in this application can also be implemented in the form of a computer program product, which includes a computer program. When this computer program product runs on a computer device, the computer program is used to cause the computer device to execute the steps in the item rental status detection method according to various exemplary embodiments of this application described above in this specification. For example, the computer device can execute the steps of each embodiment.

[0258] The computer program product may employ any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0259] The computer program product of the embodiments of the present application may employ a portable compact disk read-only memory (CD-ROM) and include a computer program, and may be run on a computer device. However, the computer program product of the present application is not limited thereto. In the present application, the readable storage medium may be any tangible medium that contains or stores a program, and the computer program included therein may be used by or in conjunction with a command execution system, apparatus, or device.

[0260] The readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a readable computer program. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0261] The computer program contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the foregoing.

[0262] The computer program for performing the operations of the present application may be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages.

[0263] It should be noted that although several units or subunits of the apparatus are mentioned in the foregoing detailed description, such a division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more of the above-described units may be embodied in one unit. Conversely, the features and functions of one unit described above may be further divided and embodied by multiple units.

[0264] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the shown operations must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution.

[0265] Those skilled in the art should understand that the embodiments of the present application may be provided as a method, a system, or a computer program product. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer programs.

[0266] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications that fall within the scope of the present application.

[0267] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and modifications.

Claims

1. A method for detecting the rental status of an item, characterized in that, the method includes: In response to a rental operation triggered by a target object, adjusting the field of view angle of an identity recognition device associated with the item rental device from a first field of view angle to a second field of view angle, where the identity recognition device is used for identity recognition based on the captured images, and the second field of view angle is greater than the first field of view angle; Obtain a set of status detection images collected at the second field of view angle after the identity recognition device identifies the identity information of the target object. The status detection images included in the set of status detection images present at least one detection object, and the at least one detection object includes a first rental item assigned to the target object; According to the set of status detection images, determine the position change information of each of the at least one detection object between the status detection images in the set of status detection images; According to the position change information corresponding to each of the at least one detection object, determine the item rental status of the target object for the first rental item.

2. The method according to claim 1, characterized in that, the position change information includes a first actual movement trajectory of the first rental item, and the item rental status includes an unlocking status of the first rental item; then determining the item rental status of the target object for the first rental item according to the position change information corresponding to each of the at least one detection object includes: If the first actual movement trajectory matches a preset first reference movement trajectory, determine that the unlocking status of the first rental item is successful unlocking; If the first actual movement trajectory does not match the first reference movement trajectory, determine that the unlocking status of the first rental item is failed unlocking.

3. The method according to claim 2, characterized in that, the set of status detection images includes a first image subset, and the first image subset includes a reference image and an image sequence at an image acquisition time after the reference image; then determining the position change information of each of the at least one detection object between the status detection images in the set of status detection images includes: According to the first item information corresponding to the first rental item, determine the original position of the first rental item in the reference image; According to the first item information and the original position, determine the position sequence of the first rental item in the image sequence, where the position sequence includes the actual positions of the first rental item in each image in the image sequence; According to the original position and the position sequence, determine the first actual movement trajectory.

4. The method according to claim 1, characterized in that, the item rental status includes an unlocking status of the first rental item, and the set of status detection images includes a first image subset, and the first image subset includes a reference image and an image sequence at an image acquisition time after the reference image; then determining the item rental status of the target object for the first rental item according to the position change information corresponding to each of the at least one detection object includes: Determine the original position of the first leased item in the reference image according to the first item information corresponding to the first leased item; Extract the reference pixel features corresponding to the original position from the reference image; Extract the target pixel features corresponding to the original position from each image in the image sequence respectively; Determine the unlocking state according to the feature differences between the reference pixel features and the obtained target pixel features; 5. The method according to claim 2, wherein, the position change information further includes a second actual movement trajectory of the limb of the target object, and the item rental state includes the removal state of the first leased item; After determining that the unlocking state of the first leased item is successfully unlocked if the first actual movement trajectory of the first leased item matches a preset first reference movement trajectory, the method further includes: If the second actual movement trajectory meets the preset conditions, determine that the removal state of the first leased item is successfully removed; If the second actual movement trajectory does not meet the preset conditions, determine that the unlocking state of the first leased item is a removal failure; wherein, the preset conditions include: the second actual movement trajectory matches the first actual movement trajectory, and the distance between the limb and the first leased item in each image corresponding to the second actual movement trajectory is not greater than a preset threshold.

6. The method according to claim 2, wherein, the state detection image set includes a second image subset, and the item rental state includes the removal state of the first leased item; After determining that the unlocking state of the first leased item is successfully unlocked if the first actual movement trajectory of the first leased item matches a preset first reference movement trajectory, the method further includes: Input each image included in the second image subset into a trained image recognition model to obtain an image recognition result corresponding to each image, and the image recognition result is used to indicate the interaction type between the limb and the first leased item in the corresponding image; Determine the removal state according to the obtained image recognition results; wherein, the state detection model is trained based on a plurality of training samples, and each training sample in the plurality of training samples includes a sample image and its corresponding training label, and the training label is used to indicate the interaction type between the limb and the leased item.

7. The method according to claim 2, wherein, the state detection image set includes a second image subset, and the item rental state includes the removal state of the first leased item; After determining that the unlocking state of the first leased item is successfully unlocked if the first actual movement trajectory of the first leased item matches a preset first reference movement trajectory, the method further includes: For each state detection image included in the second image subset, perform the following operations respectively: For a status detection image, input the status detection image into a trained image segmentation model to obtain an image segmentation result corresponding to the status detection image, where the image segmentation result includes the position information of the limbs of the target object and the first rental item in the status detection image; Determine the relative position information between the limb and the first rental item according to the position information of the limb and the first rental item respectively; Determine the removal status according to the relative position information corresponding to each status detection image and a preset position constraint condition.

8. The method according to any one of claims 1 to 7, characterized in that, after determining the item rental status of the target object for the first rental item according to the position change information corresponding to each of the at least one detection object, the method further includes: if the item rental status is unlocking failure or removal failure, perform at least one of the following operations: reassign a second rental item for the target object from the rental items in the idle state; generate an alarm message based on the item rental status and send the alarm message to the background server.

9. The method according to any one of claims 1 to 7, characterized in that, the response to the rental operation of the target object to adjust the field of view angle of the identity recognition device associated with the item rental device from a first field of view angle to a second field of view angle includes any one of the following methods: in response to the rental operation, send an adjustment instruction to the identity recognition device, where the adjustment instruction is used to instruct the identity recognition device to adjust the field of view angle to the second field of view angle; in response to the rental operation, collect an identity recognition image of the target object at the first field of view angle, and if the identity information of the target object is successfully recognized according to the identity recognition image, send the adjustment instruction to the identity recognition device; in response to the rental operation, after recognizing the identity information of the target object, if the first rental item is successfully allocated to the target object, send the adjustment instruction to the identity recognition device.

10. The method according to claim 9, characterized in that, the identity recognition device is a palm brushing device, and the identity recognition image is a palm image; then after collecting the identity recognition image of the target object at the first field of view angle in response to the rental operation, the method further includes: extract features from the palm image to obtain the target palm features of the target object; match the target palm features with at least one reference palm feature to determine the reference palm feature that matches the target palm features; determine the identity information associated with the reference palm feature as the identity information of the target object.

11. An item rental status detection device, characterized in that, the device includes: an adjustment unit, configured to respond to a rental operation triggered by a target object, and adjust the field of view angle of an identity recognition device associated with an item rental device from a first field of view angle to a second field of view angle, where the identity recognition device is used for identity recognition according to the collected image, and the second field of view angle is greater than the first field of view angle; An image acquisition unit, configured to obtain a state detection image set acquired at the second field of view angle after the identity recognition device recognizes the identity information of the target object, where at least one detection object is presented in the state detection images included in the state detection image set, and the at least one detection object includes a first rental item assigned to the target object; An image processing unit, configured to determine, according to the state detection image set, position change information of each of the at least one detection object between the state detection images in the state detection image set; A state determination unit, configured to determine an item rental state of the target object for the first rental item according to the position change information corresponding to each of the at least one detection object.

12. A computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, when the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product, including a computer program, wherein, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images