In-place state determination method and apparatus, storage medium, and processor

Abstract

The application discloses a seat state determination method and device, a storage medium and a processor. The seat state determination method comprises the following steps: acquiring position information of a seat area and a target image, wherein the target image is an image of a preset area; and determining whether a passenger is on the seat according to the position information of the seat area and the target image. The seat state determination method solves the problem that the in-seat state of a person is not accurately determined in the prior art.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically, to a method, apparatus, storage medium, and processor for determining in-situ state. Background Technology

[0002] With urban development and the improvement of residents' living standards, the demand for various types of vehicles is increasing day by day. This has led to technological innovation and industrial upgrading around how to ensure the safety of passengers during the ride. Among these efforts, the status information of passengers has received more attention, such as determining whether the driver is in the driver's seat and whether the passengers are in their seats.

[0003] Drivers often experience fatigue due to the need for continuous, high concentration while driving, which can easily lead to distraction and ultimately traffic accidents. While the government has enacted laws and regulations limiting continuous driving time, some drivers still exceed these limits for greater profit. Furthermore, in most safety-critical commercial vehicles, passengers moving around in the relatively confined space or not being seated can distract the driver and even endanger others during sudden braking or turns. Therefore, a reasonable method is needed to provide accurate and comprehensive information on the occupants' positions to prevent potential accidents caused by abnormal occupant status.

[0004] Currently, mature presence detection solutions typically use the detection results of a pressure sensor installed under the driver's seat to determine whether a person is present. This mechanism increases vehicle production costs due to the addition of physical components, requires significant modifications to the internal wiring, hinders the scalability of the entire presence detection solution, and is prone to errors due to items placed on the seat.

[0005] In addition, some related technologies indirectly determine a person's presence by detecting the presence of a face. The core logic is that if a face is detected, the person is definitely present; if no face is detected, the person is definitely not present. However, in reality, neither of these two logics necessarily holds true, resulting in a high rate of missed or false detections.

[0006] There is currently no effective solution to the above problems.

[0007] The information disclosed in the background section is only intended to enhance the understanding of the background art described herein. Therefore, the background art may contain information that does not constitute prior art to those skilled in the art. Summary of the Invention

[0008] This invention provides a method, apparatus, storage medium, and processor for determining a person's presence status, thereby at least addressing the problem of inaccurate determination of a person's presence status in the prior art.

[0009] To achieve the above objective, according to a first aspect of the present invention, a method for determining a passenger's position status is provided, comprising: acquiring location information of a seating area and a target image, wherein the target image is an image of a preset area; and determining whether a passenger is in a seat based on the location information of the seating area and the target image.

[0010] Furthermore, based on the location information of the seat area and the target image, it is determined whether the passenger is in the seat, including: identifying the part of the target image corresponding to the location information of the seat area as the key image; inputting the key image into the pre-trained presence judgment model to obtain the judgment result of whether the passenger is in the seat.

[0011] Furthermore, acquiring the location information and target image of the seating area includes: acquiring a reference image of the passenger sitting in the seat; determining the location information of the face area based on the reference image; and determining the location information of the seating area based on the location information of the face area.

[0012] Further, determining the location information of the seating area based on the location information of the face region includes: controlling at least one boundary of the face region to extend by a preset length in at least one direction to obtain the seating area.

[0013] Furthermore, the location information of the seat area is determined based on the location information of the face area, including: extending the face area upward by a first length as the upper boundary of the seat area; extending the face area downward by a second length as the lower boundary of the seat area; and extending the face area to the left and right by a third length as the two side boundaries of the seat area.

[0014] Further, determining whether a passenger is in a seat based on the location information of the seat area and the target image includes: acquiring the location information of the head and torso region in the target image; comparing the location information of the head and torso region with the location information of the seat area, and determining whether the passenger is in a seat based on the comparison result, wherein the location information of the seat area is a preset value.

[0015] Furthermore, obtaining the location information of the human head and body region in the target image includes: inputting the target image into a pre-trained head and body detection model to obtain the location information of the human head and body region.

[0016] Further, determining whether a passenger is in a seat based on the location information of the seat area and the target image includes: obtaining the center point of the human body area in the target image; comparing the center point of the human body area with the center point of the seat area, and determining whether the passenger is in a seat based on the comparison result, wherein the location information of the seat area is a preset value, and the seat area is the area determined based on the location information of the seat area.

[0017] Further, obtaining the center point of the human body region in the target image includes: inputting the target image into a pre-trained estimation model to obtain the confidence value of the presence of a passenger at each pixel in the target image; removing pixels in the target image with confidence values ​​less than a set value; clustering the remaining pixels in the target image and using the obtained cluster center as the center point of the human body region.

[0018] Furthermore, the training method for the estimation model includes setting up an initial model based on a convolutional neural network and a training step that is executed iteratively. The training step includes: inputting training images into the initial model, wherein the training images include pixels with a pre-labeled confidence value of 1; comparing the pixel with the highest confidence value output by the initial model with the pre-labeled pixels with a confidence value of 1, and updating the parameters of the initial model based on the comparison result.

[0019] Further, acquiring the location information of the seating area and the target image includes: acquiring the target image and detecting the location information of the seating area based on the target image; determining whether a passenger is in a seat based on the location information of the seating area and the target image includes: detecting the location information of a face region based on the target image; if both the location information of the seating area and the location information of the face region are detected, and the relationship between the location information of the seating area and the location information of the face region satisfies a predetermined condition, then it is determined that the passenger is in a seat; if the location information of the seating area is detected but the location information of the face region is not detected, or .... If the location information of the seat region and the location information of the face region do not meet the predetermined conditions, the part of the target image corresponding to the location information of the seat region is input into the pre-trained presence judgment model to obtain the judgment result of whether the passenger is in the seat; if neither the location information of the seat region nor the location information of the face region is detected, the location information of the historical seat region is used as the location information of the seat region, and the part of the target image corresponding to the location information of the seat region is input into the pre-trained presence judgment model to obtain the judgment result of whether the passenger is in the seat. Here, the location information of the historical seat region is the location information of the seat region determined based on the image of the target image at the previous moment.

[0020] Furthermore, if the center point of the face region is located within the reference region, then the relationship between the location information of the seat region and the location information of the face region satisfies a predetermined condition; if the center point of the face region is located outside the reference region, then the relationship between the location information of the seat region and the location information of the face region does not satisfy the predetermined condition; wherein, the reference region is a part of the seat region, the face region is the region determined based on the location information of the face region, and the seat region is the region determined based on the location information of the seat region.

[0021] Furthermore, the seating area is a first rectangular area with a width of a and a height of b; the reference area is a second rectangular area with a width of a / 2 and a height of a / 3; the top edge of the second rectangular area coincides with the top edge of the first rectangular area, and the distances from both sides of the second rectangular area along its width direction to the corresponding sides of the first rectangular area along its width direction are equal.

[0022] According to a second aspect of the present invention, an on-site status determination device is provided, comprising: an acquisition unit, configured to acquire location information of a seating area and a target image, wherein the target image is an image of a preset area; and a determination unit, configured to determine whether a passenger is in a seat based on the location information of the seating area and the target image.

[0023] According to a third aspect of the present invention, a non-volatile storage medium is provided, the non-volatile storage medium including a stored program, wherein, when the program is running, it controls the device where the non-volatile storage medium is located to execute the above-described in-situ state determination method.

[0024] According to a fourth aspect of the present invention, a processor is provided for running a program, wherein the program executes the above-described method for determining the in-situ state during runtime.

[0025] The method for determining the passenger's position using the technical solution of this invention includes: acquiring location information of a seating area and a target image, and determining whether a passenger is in a seat based on the location information of the seating area and the target image. By acquiring the seating area and combining it with the determined seating area, the acquired target image can be judged more specifically, thereby determining whether a passenger is in a seat. This approach helps to eliminate redundant interference factors in the target image, improves the accuracy of the position determination, and solves the problem of low accuracy in existing position determination methods. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0027] Figure 1 This is a flowchart illustrating an optional embodiment of the method for determining the in-situ status according to the present invention;

[0028] Figure 2 This is a flowchart illustrating the first specific embodiment of the method for determining the in-situ status according to the present invention;

[0029] Figure 3 This is a flowchart illustrating a second specific embodiment of the method for determining the in-situ status according to the present invention;

[0030] Figure 4 This is a flowchart illustrating a third specific embodiment of the method for determining the in-situ status according to the present invention;

[0031] Figure 5 This is a flowchart illustrating the fourth specific embodiment of the method for determining the in-situ status according to the present invention;

[0032] Figure 6 This is a schematic diagram of an optional embodiment of the in-situ status determination device according to the present invention. Detailed Implementation

[0033] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] It should be understood that when an element (such as a layer, film, region, or substrate) is described as being "on" another element, the element may be directly on the other element, or there may be an intermediate element present. Furthermore, in the specification and claims, when an element is described as being "connected" to another element, the element may be "directly connected" to the other element, or "connected" to the other element via a third element.

[0037] Figure 1 This is an embodiment of the present invention for determining the in-situ status, such as... Figure 1 As shown, the method includes the following steps:

[0038] Step S101: Obtain the location information and target image of the seating area, wherein the target image is an image of a preset area;

[0039] Step S102: Determine whether the passenger is in a seat based on the location information of the seating area and the target image.

[0040] The on-seat status determination method of this invention includes: acquiring location information of a seat area and a target image, and determining whether a passenger is in a seat based on the location information of the seat area and the target image. By acquiring the seat area and combining it with the determined seat area, the acquired target image can be judged more specifically, thereby determining whether a passenger is in a seat. This approach helps to eliminate redundant interference factors in the target image, improves the accuracy of on-seat status determination, and solves the problem of low accuracy in existing on-seat status determination methods.

[0041] like Figure 2 As shown, in one embodiment, determining whether a passenger is in a seat based on the location information of the seat area and the target image includes: determining the portion of the target image corresponding to the location information of the seat area as a key image; inputting the key image into a pre-trained presence determination model to obtain a determination result on whether the passenger is in a seat.

[0042] By extracting the portion of the target image corresponding to the seat area and inputting it into a pre-trained presence determination model, the model can determine whether a passenger is present based on the input image information. Because the portion of the image corresponding to the seat area is input separately into the presence determination model, it can make more targeted judgments, which helps improve accuracy and reduce computational load.

[0043] Specifically, acquiring the location information and target image of the seating area includes: acquiring a reference image of the passenger sitting in the seat; determining the location information of the face area based on the reference image; and determining the location information of the seating area based on the location information of the face area.

[0044] By acquiring a reference image of a passenger sitting in a normal position, the face region of the passenger in a normal sitting posture is determined based on the reference image, and the theoretical seat region is estimated using this as a reference. In this way, the seat region can more realistically reflect the area on the target image that is highly related to whether the passenger is in place. Using this as a reference to determine whether the passenger is in place can effectively improve the accuracy of the judgment.

[0045] Specifically, the location information of the seat area is determined based on the location information of the face area, including: extending the face area upward by a first length as the upper boundary of the seat area; extending the face area downward by a second length as the lower boundary of the seat area; and extending the face area to the left and right by a third length as the two side boundaries of the seat area.

[0046] In this way, the boundaries of the seating area can be determined more easily and reasonably based on the face region, thus facilitating the subsequent determination of the seat's position.

[0047] In practice, based on the face region, at least one boundary of the face region can be extended by a preset length in at least one direction to obtain the seating area.

[0048] For example, the first length can be half the height of the face region, the second length can be twice the height of the face region, and the third length can be equal to the length of the face region. Of course, if high accuracy is not required, the first, second, and third lengths can also take other values.

[0049] In practice, the step of determining the face region based on the reference image can be achieved in various ways. For example, it can be determined by a dedicated face detection algorithm or by a pre-trained face recognition model.

[0050] like Figure 3 As shown, in another embodiment, determining whether a passenger is in a seat based on the location information of the seat area and the target image includes: acquiring the location information of the head and torso region in the target image; comparing the location information of the head and torso region with the location information of the seat area, and determining whether the passenger is in a seat based on the comparison result, wherein the location information of the seat area is a preset value.

[0051] By acquiring the head and torso region and comparing it with the seat region, the relative position of the passenger and the seat can be determined based on the comparison result, thus determining whether the passenger is in their seat.

[0052] Specifically, obtaining the location information of the human head and body region in the target image includes: inputting the target image into a pre-trained head and body detection model to obtain the location information of the human head and body region.

[0053] By employing a trained head and body detection model to detect the head and body region in the target image, this region can be obtained relatively easily and accurately. By comparing the obtained head and body region with a preset seat area, the passenger's presence can be accurately determined based on the comparison result. Several methods can be used to determine whether a passenger is seated based on the comparison result. For example, if the area of ​​the overlapping region between the obtained head and body region and the seat area reaches a set proportion, the passenger is determined to be seated. Another example is if the proportion of the portion of the head and body region within the seat area to the total area of ​​the head and body region reaches a set value, the passenger is determined to be seated. Yet another example is if the distance between the geometric center of the head and body region and the geometric center of the seat area is less than a set value, the passenger is determined to be seated.

[0054] like Figure 4 As shown, in another specific embodiment, determining whether a passenger is in a seat based on the location information of the seat area and the target image includes: obtaining the center point of the human body area in the target image; comparing the center point of the human body area with the center point of the seat area, and determining whether the passenger is in a seat based on the comparison result, wherein the location information of the seat area is a preset value, and the seat area is the area determined based on the location information of the seat area.

[0055] By pre-setting the seating area, once the center point of the human body area in the target image is obtained, the center point of the human body area is compared with the center point of the seating area. This can objectively reflect the relative positional relationship between the human body and the seat, thus enabling a more accurate and convenient judgment on whether the passenger is in their seat.

[0056] In this embodiment, obtaining the center point of the human body region in the target image includes: inputting the target image into a pre-trained estimation model to obtain the confidence value of the presence of a passenger at each pixel in the target image; removing pixels in the target image with confidence values ​​less than a set value; clustering the remaining pixels in the target image and using the obtained cluster center as the center point of the human body region.

[0057] By employing an estimation model to process the input target image, the confidence value of the presence of passengers at each pixel of the target image is obtained. The confidence value, or probability value, represents the likelihood of a passenger being present at that pixel. By identifying pixels with confidence values ​​less than a set value, the remaining pixels can better represent the location information and distribution of passengers. Furthermore, by clustering the remaining pixels, the center point of the human body region can be obtained more accurately, thus providing a more accurate and reliable basis for the subsequent determination of the passenger's presence status.

[0058] In practice, the process of clustering the remaining pixels in the target image can be carried out using various clustering methods, such as the K-means method and the KNN method.

[0059] Specifically, the training method for the estimation model includes setting up an initial model based on a convolutional neural network and a training step that is executed cyclically. The training step includes: inputting training images into the initial model, wherein the training images include pixels with a pre-labeled confidence value of 1; comparing the pixel with the highest confidence value output by the initial model with the pre-labeled pixel with a confidence value of 1, and updating the parameters of the initial model based on the comparison result.

[0060] In one specific embodiment, a training mask is created based on the training image, with the same resolution as the training image and marking whether a person exists in the training image. Pixels with a pre-marked confidence value of 1 in the training mask are the center points where a person exists in the training image, while the confidence values ​​of the remaining pixels are either set to 0 or set to values ​​with high distinguishability from 1 according to a certain strategy. The training mask is then input into the initial model. The mask output obtained by the initial model through the training image input is compared with the training mask, and the parameters of the initial model are updated according to the comparison results until the difference in confidence distribution between the model output mask and its corresponding training mask is less than a preset threshold.

[0061] A pixel with a confidence value of 1 is the geometric center of at least a portion of the body of each passenger in the target image. For example, in this embodiment, a pixel with a confidence value of 1 is the geometric center of the head and torso of each passenger in the target image.

[0062] like Figure 5As shown, in another specific embodiment, obtaining the location information of the seating area and the target image includes: obtaining the target image and detecting the location information of the seating area based on the target image; determining whether a passenger is in a seat based on the location information of the seating area and the target image includes: detecting the location information of a face area based on the target image; if both the location information of the seating area and the location information of the face area are detected, and the relationship between the location information of the seating area and the location information of the face area satisfies a predetermined condition, then it is determined that the passenger is in a seat; if the location information of the seating area is detected but the location information of the face area is not detected, or both the location information of the seating area and the location information of the face area are detected... If a passenger is detected and the location information of the seat area and the location information of the face area do not meet a predetermined condition, then the portion of the target image corresponding to the location information of the seat area is input into the pre-trained presence judgment model to obtain a judgment result on whether the passenger is in the seat. If neither the location information of the seat area nor the location information of the face area is detected, then the location information of the historical seat area is used as the seat area, and the portion of the target image corresponding to the location information of the seat area is input into the pre-trained presence judgment model to obtain a judgment result on whether the passenger is in the seat. Here, the location information of the historical seat area is the location information of the seat area determined based on the image of the target image at the previous moment.

[0063] In other words, the steps of acquiring the location information of the seating area and the target image involve acquiring the target image and detecting the seating area based on the target image. The process of determining whether a passenger is seated based on the location information of the seating area and the target image involves face detection on the target image.

[0064] The historical seating region is the seating region determined based on images from previous moments before the target image. In other words, the historical seating region is the seating region determined based on images acquired at the previous moment or on average from several previous moments, meaning that the seating region exists in the image at that moment and has been successfully detected.

[0065] In other words, after acquiring the target image, the system detects the positional information of the seat area and the face area within the target image. Based on the detection results, it further determines whether the passenger is in their seat. When both the seat area and the face area are detected, and the relationship between their positional information satisfies predetermined conditions, it can be directly determined that the passenger is in their seat. When no face area is detected, or the positional relationship between the detected seat area and the detected face area does not meet predetermined conditions, it is impossible to directly determine whether the passenger is in their seat. In this case, the portion of the target image corresponding to the seat area is input separately into the in-seat judgment model, which automatically determines the passenger's in-seat status. In extreme cases, if neither the face area nor the seat area is detected, the seat area determined from the image acquired at the previous moment can be used as the seat area of ​​the target image, and the portion of the target image corresponding to this area is input into the in-seat judgment model, thereby outputting the in-seat judgment result. This allows for the selection of different judgment methods based on different actual situations, which helps improve judgment speed and accuracy. Because the method used in the in-situ judgment model is to input the part of the target image corresponding to the seat area separately into the in-situ judgment model, it can perform more targeted detection and judgment, which helps to reduce the amount of computation and reduce interference from other areas, thereby improving the judgment accuracy.

[0066] Specifically, if the center point of the face region is located within the reference region, then the relationship between the location information of the seat region and the location information of the face region satisfies a predetermined condition; if the center point of the face region is located outside the reference region, then the relationship between the location information of the seat region and the location information of the face region does not satisfy the predetermined condition; wherein, the reference region is a part of the seat region, the face region is the region determined based on the location information of the face region, and the seat region is the region determined based on the location information of the seat region.

[0067] As can be seen, in this embodiment, the method for judging the predetermined conditions is to judge the relationship between the center point of the face region and the reference region. If the center point of the face region is within the reference region, the passenger can be reliably determined to be seated.

[0068] To more reasonably determine the specific range of the reference area and thus improve the accuracy of the seat judgment, the seat area is a first rectangular area with a width of 'a' and a height of 'b'; the reference area is a second rectangular area with a width of 'a / 2' and a height of 'a / 3'; the top edge of the second rectangular area coincides with the top edge of the first rectangular area, and the distances from both sides of the second rectangular area along its width direction to the corresponding sides of the first rectangular area along its width direction are equal.

[0069] In other words, the center line of the second rectangular region along its width coincides with the center line of the first rectangular region along its width. That is, the second rectangular region is centered relative to the first rectangular region along the width direction of both.

[0070] In practical applications, the on-site status determination method of the embodiments of the present invention can be used in vehicles, ships, aircraft and other vehicles, as well as in other places where there are seats, such as offices, restaurants and so on. The solution of the embodiments of the present invention can conveniently and accurately determine the on-site status of personnel, thereby facilitating subsequent management, control or service processes.

[0071] Specifically, in the above steps, the process of determining the face region, seat region, head and torso region based on the image can be achieved by using image recognition technology to obtain the corresponding adaptive region of interest.

[0072] like Figure 6 As shown, the present invention also provides an on-site status determination device, comprising: an acquisition unit for acquiring location information of a seating area and a target image, wherein the target image is an image of a preset area; and a determination unit for determining whether a passenger is in a seat based on the location information of the seating area and the target image.

[0073] The on-seat status determination device of this invention includes an acquisition unit and a determination unit. The acquisition unit acquires location information of a seating area and a target image; the determination unit determines whether a passenger is in a seat based on the location information of the seating area and the target image. By acquiring the seating area through the acquisition unit, the determination unit can more effectively judge the acquired target image based on the determined seating area, thereby determining whether a passenger is in a seat. This helps to eliminate redundant interference factors in the target image, improves the accuracy of on-seat status determination, and solves the problem of low accuracy in existing on-seat status determination devices.

[0074] Specifically, the determining unit includes a first determining module and a first input module. The first determining module is used to determine the part of the target image corresponding to the position information of the seat area as the key image. The first input module is used to input the key image into the pre-trained in-situ judgment model to obtain the judgment result of whether the passenger is in the seat.

[0075] By extracting the portion of the target image corresponding to the seat area and inputting it into a pre-trained presence determination model, the model can determine whether a passenger is present based on the input image information. Because the portion of the image corresponding to the seat area is input separately into the presence determination model, it can make more targeted judgments, which helps improve accuracy and reduce computational load.

[0076] Specifically, the acquisition unit includes a first acquisition module, a second determination module, and a third determination module. The first acquisition module is used to acquire a reference image of the passenger sitting in the seat; the second determination module is used to determine the position information of the face region based on the reference image; and the third determination module is used to determine the position information of the seat region based on the position information of the face region.

[0077] By acquiring a reference image of a passenger sitting in a normal position, the face region of the passenger in a normal sitting posture is determined based on the reference image, and the theoretical seat region is estimated using this as a reference. In this way, the seat region can more realistically reflect the area on the target image that is highly related to whether the passenger is in place. Using this as a reference to determine whether the passenger is in place can effectively improve the accuracy of the judgment.

[0078] Specifically, the third determining module is used to: extend the face region upwards by a first length as the upper boundary of the seating area; extend the face region downwards by a second length as the lower boundary of the seating area; and extend the face region to the left and right by a third length as the two side boundaries of the seating area.

[0079] In this way, the boundaries of the seating area can be determined more easily and reasonably based on the facial region, thus facilitating the subsequent on-seat determination process.

[0080] In this embodiment, the first length is half the height of the face region, the second length is twice the height of the face region, and the third length is equal to the length of the face region. Of course, the first, second, and third lengths can also take other values, but the accuracy of the determined seating area will decrease.

[0081] In practice, the second determination module can be implemented in various ways. The most common way is to determine it through a dedicated face detection algorithm or by using a pre-trained face recognition model.

[0082] Specifically, the determining unit includes a second acquisition module and a first comparison module; the second acquisition module is used to acquire the position information of the head and body region in the target image; the first comparison module is used to compare the position information of the head and body region with the position information of the seat region, and determine whether the passenger is in the seat based on the comparison result, wherein the position information of the seat region is a preset value.

[0083] By acquiring the head and torso region and comparing it with the seat region, the relative position of the passenger and the seat can be determined based on the comparison result, thus determining whether the passenger is in their seat.

[0084] In practical implementation, the first comparison module can determine whether a passenger is seated in various ways. For example, it can determine the proportion of the overlap between the head and torso area and the seat area within the seat area; if this proportion is greater than a set value, it determines that the passenger is seated. Another example is determining the proportion of the head and torso area within the seat area; if this proportion is greater than a set value, it determines that the passenger is seated.

[0085] Specifically, the second acquisition module is used to input the target image into the pre-trained head and body detection model to obtain the location information of the human head and body region.

[0086] By employing a trained head and body detection model to detect the head and body region in a target image, this region can be obtained relatively easily and accurately. By comparing the obtained head and body region with a preset seat area, the passenger's seating position can be accurately determined based on the comparison result. For example, if the area of ​​the overlapping region between the obtained head and body region and the seat area reaches a set proportion, the passenger is determined to be seated. Several implementation methods can be used to determine whether a passenger is seated based on the comparison result. For instance, if the proportion of the portion of the head and body region within the seat area to the total area of ​​the head and body region reaches a set value, the passenger is determined to be seated. Another example is if the distance between the geometric center of the head and body region and the geometric center of the seat area is less than a set value, the passenger is determined to be seated.

[0087] Specifically, the determining unit includes a third acquisition module and a second comparison module; the third acquisition module is used to acquire the center point of the human body region in the target image; the second comparison module is used to compare the center point of the human body region with the center point of the seat region, and determine whether the passenger is in the seat based on the comparison result, wherein the position information of the seat region is a preset value, and the seat region is the region determined based on the position information of the seat region.

[0088] By pre-setting the seating area, once the center point of the human body area in the target image is obtained, that is, by comparing the center point of that human body area with the center point of the seating area, the relative positional relationship between the human body and the seat can be reflected more objectively, thus enabling a more accurate and convenient judgment on whether the passenger is in place.

[0089] In this embodiment, the third acquisition module includes an input submodule, a removal submodule, and a clustering submodule: the input submodule is used to input the target image into a pre-trained estimation model to obtain the confidence value of the presence of passengers at each pixel in the target image; the removal submodule is used to remove pixels in the target image with confidence values ​​less than a set value; the clustering submodule is used to cluster the remaining pixels in the target image and use the obtained cluster center as the center point of the human body region.

[0090] By employing an estimation model to process the input target image, the confidence value of the presence of passengers at each pixel of the target image is obtained. The confidence value, or probability value, represents the likelihood of a passenger being present at that pixel. By identifying pixels with confidence values ​​less than a set value, the remaining pixels can better represent the location information and distribution of passengers. Furthermore, by clustering the remaining pixels, the center point of the human body region can be obtained more accurately, thus providing a more accurate and reliable basis for the subsequent determination of the passenger's presence status.

[0091] In practice, the process of clustering the remaining pixels in the target image can be carried out using various clustering methods, such as K-means or KNN.

[0092] Specifically, the training method for the estimation model includes setting up an initial model based on a convolutional neural network and a training step that is executed cyclically. The training step includes: inputting training images into the initial model, wherein the training images include pixels with a pre-labeled confidence value of 1; comparing the pixel with the highest confidence value output by the initial model with the pre-labeled pixel with a confidence value of 1, and updating the parameters of the initial model based on the comparison result.

[0093] A pixel with a confidence value of 1 is the geometric center of at least a portion of the body of each passenger in the target image. For example, in this embodiment, a pixel with a confidence value of 1 is the geometric center of the head and torso of each passenger in the target image.

[0094] Specifically, the acquisition unit includes a fourth acquisition module and a first detection module: the fourth acquisition module is used to acquire the target image, and the first detection module is used to detect the position information of the seat area based on the target image; the determination unit includes a second detection module and a processing module: the second detection module is used to detect the position information of the face area based on the target image; the processing module is used to determine whether the passenger is in the seat. If both the position information of the seat area and the position information of the face area are detected, and the relationship between the position information of the seat area and the position information of the face area meets a predetermined condition, then the processing module determines that the passenger is in the seat; if the position information of the seat area is detected but the position information of the face area is not detected, or the position information of the seat area and the position information of the face area are not detected, then the processing module determines that the passenger is in the seat; If all information is detected, and the location information of the seat area and the location information of the face area do not meet the predetermined conditions, the processing module inputs the part of the target image corresponding to the location information of the seat area into the pre-trained presence judgment model to obtain the judgment result of whether the passenger is in the seat; if neither the location information of the seat area nor the location information of the face area is detected, the processing module uses the location information of the historical seat area as the location information of the seat area, inputs the part of the target image corresponding to the location information of the seat area into the pre-trained presence judgment model to obtain the judgment result of whether the passenger is in the seat, wherein the location information of the historical seat area is the location information of the seat area determined based on the image of the target image at the previous moment.

[0095] The location information of the historical seat area is the location information of the seat area determined based on the image of the target image at the previous moment. In other words, the historical seat area is the seat area determined based on the image acquired at the previous moment, meaning that the seat area exists in the image at that moment and is successfully detected.

[0096] In other words, after acquiring the target image, the system detects the positional information of the seat area and the face area within the target image. Based on the detection results, it further determines whether the passenger is in their seat. Specifically, if both the seat area and the face area are detected, and the relationship between their positional information satisfies predetermined conditions, it can be directly determined that the passenger is in their seat. If no face area is detected, or the positional relationship between the detected seat area and the detected face area does not meet predetermined conditions, it is impossible to directly determine whether the passenger is in their seat. In this case, the portion of the target image corresponding to the seat area is input separately into the in-seat judgment model, which automatically determines the passenger's in-seat status. In extreme cases, if neither the face area nor the seat area is detected, the seat area determined from the image acquired at the previous moment can be used as the seat area of ​​the target image, and the portion of the target image corresponding to this area is input into the in-seat judgment model, thus outputting the in-seat judgment result. This allows for the selection of different judgment methods based on different actual situations, which helps improve judgment speed and accuracy. Because the method used in the in-situ judgment model is to input the part of the target image corresponding to the seat area separately into the in-situ judgment model, it can perform more targeted detection and judgment, which helps to reduce the amount of computation and reduce interference from other areas, thereby improving the judgment accuracy.

[0097] Specifically, if the center point of the face region is located within the reference region, the processing module determines that the relationship between the location information of the seat region and the location information of the face region meets a predetermined condition; if the center point of the face region is located outside the reference region, the processing module determines that the relationship between the location information of the seat region and the location information of the face region does not meet the predetermined condition; wherein, the reference region is a part of the seat region, the face region is the region determined based on the location information of the face region, and the seat region is the region determined based on the location information of the seat region.

[0098] As can be seen, in this embodiment, the method for judging the predetermined conditions is to judge the relationship between the center point of the face region and the reference region. If the center point of the face region is within the reference region, the passenger can be reliably determined to be seated.

[0099] To more reasonably determine the specific range of the reference area and thus improve the accuracy of the seat judgment, the seat area is a first rectangular area with a width of 'a' and a height of 'b'; the reference area is a second rectangular area with a width of 'a / 2' and a height of 'a / 3'; the top edge of the second rectangular area coincides with the top edge of the first rectangular area, and the distances from both sides of the second rectangular area along its width direction to the corresponding sides of the first rectangular area along its width direction are equal.

[0100] In other words, the center line of the second rectangular region along its width coincides with the center line of the first rectangular region along its width. That is, the second rectangular region is centered relative to the first rectangular region along the width direction of both.

[0101] Furthermore, the present invention also provides a non-volatile storage medium, which includes a stored program, wherein, when the program is running, it controls the device where the non-volatile storage medium is located to execute the above-described in-situ state determination method.

[0102] In addition, the present invention also provides a processor for running a program, wherein the program executes the above-described method for determining the in-situ state during runtime.

[0103] The sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. Moreover, the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than that shown here.

[0104] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0106] The units described as separate components may or may not be physically separate. Similarly, the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0107] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0108] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0109] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for determining the in-situ state, characterized in that, include: Acquire the location information and target image of the seating area, wherein the target image is an image of a preset area; Based on the location information of the seating area and the target image, determine whether the passenger is in a seat. Acquiring the location information of the seating area and a target image includes: acquiring the target image and detecting the location information of the seating area based on the target image; Determining whether a passenger is in a seat based on the location information of the seating area and the target image includes: detecting the location information of a face region based on the target image; if both the location information of the seating area and the location information of the face region are detected, and the relationship between the location information of the seating area and the location information of the face region satisfies a predetermined condition, then the passenger is determined to be in a seat; if the location information of the seating area is detected but the location information of the face region is not detected, or if both the location information of the seating area and the location information of the face region are detected, and the relationship between the location information of the seating area and the location information of the face region is not satisfied with a predetermined condition, then the passenger is determined to be in a seat; If the predetermined conditions are met, the portion of the target image corresponding to the location information of the seat area is input into the pre-trained presence judgment model to obtain a judgment result on whether the passenger is in the seat; if neither the location information of the seat area nor the location information of the face area is detected, the location information of the historical seat area is used as the location information of the seat area, and the portion of the target image corresponding to the location information of the seat area is input into the pre-trained presence judgment model to obtain a judgment result on whether the passenger is in the seat, wherein the location information of the historical seat area is the location information of the seat area determined based on the image of the target image at the previous moment.

2. The method for determining the in-situ status according to claim 1, characterized in that, Determining whether a passenger is in a seat based on the location information of the seating area and the target image includes: The portion of the target image corresponding to the location information of the seating area is identified as the key image; The key image is input into a pre-trained presence determination model to obtain a determination result on whether the passenger is in their seat.

3. The method for determining the in-situ status according to claim 1 or 2, characterized in that, Acquire location information and target images of the seating area, including: Obtain a reference image of the passenger while seated; The location information of the face region is determined based on the reference image; The location information of the seating area is determined based on the location information of the face area.

4. The method for determining the in-situ status according to claim 3, characterized in that, Determining the location information of the seating area based on the location information of the face area includes: The seating area is obtained by extending at least one boundary of the face region by a preset length in at least one direction.

5. The method for determining the in-situ status according to claim 3, characterized in that, Determining the location information of the seating area based on the location information of the face area includes: The upper boundary of the seating area is defined by extending the face region upwards by the first length. The lower boundary of the seating area is defined by extending the face region downwards by a second length. The seating area is defined by extending the face region to the left and to the right by a third length on each side.

6. The method for determining the in-situ status according to claim 1, characterized in that, Based on the location information of the seating area and the target image, determine whether the passenger is in a seat, including: Obtain the location information of the human head and torso region in the target image; The location information of the head and torso area is compared with the location information of the seat area, and the passenger is determined to be in a seat based on the comparison result. The location information of the seat area is a preset value.

7. The method for determining the in-situ status according to claim 6, characterized in that, Obtaining the location information of the human head and torso region in the target image includes: The target image is input into a pre-trained head and body detection model to obtain the location information of the human head and body region.

8. The method for determining the in-situ status according to claim 1, characterized in that, Based on the location information of the seating area and the target image, determine whether the passenger is in a seat, including: Obtain the center point of the human body region in the target image; The center point of the human body area is compared with the center point of the seat area, and the passenger is determined to be in a seat based on the comparison result. The position information of the seat area is a preset value, and the seat area is the area determined based on the position information of the seat area.

9. The method for determining the in-situ status according to claim 8, characterized in that, Obtaining the center point of the human body region in the target image includes: The target image is input into a pre-trained estimation model to obtain the confidence value of the presence of passengers at each pixel of the target image. Remove pixels in the target image whose confidence value is less than a set value; The remaining pixels in the target image are clustered, and the obtained cluster centers are used as the center points of the human body region.

10. The method for determining the in-situ status according to claim 9, characterized in that, The training method for the estimation model includes building an initial model based on a convolutional neural network and a training step that is executed iteratively. The training step includes: The training image is input into the initial model, wherein the training image includes pre-labeled pixels with a confidence value of 1; The pixel with the highest confidence value output by the initial model is compared with the pre-labeled pixel with a confidence value of 1, and the parameters of the initial model are updated according to the comparison result.

11. The method for determining the in-situ status according to claim 1, characterized in that, If the center point of the face region is located within the reference region, then the relationship between the seat area location information and the face region location information is determined to satisfy the predetermined condition. If the center point of the face region is outside the reference region, then the relationship between the position information of the seat region and the position information of the face region does not meet the predetermined condition. Wherein, the reference area is a portion of the seating area, the face area is an area determined based on the location information of the face area, and the seating area is an area determined based on the location information of the seating area.

12. The method for determining the in-situ status according to claim 11, characterized in that, The seating area is a first rectangular area with a width of 'a' and a height of 'b'; the reference area is a second rectangular area with a width of 'a / 2' and a height of 'a / 3'; the top edge of the second rectangular area coincides with the top edge of the first rectangular area, and the distances from both sides of the second rectangular area along its width direction to the corresponding sides of the first rectangular area along its width direction are equal.

13. An in-situ status determination device, characterized in that, include: The acquisition unit is used to acquire the location information of the seating area and a target image, wherein the target image is an image of a preset area; The determining unit is configured to determine whether a passenger is in a seat based on the location information of the seating area and the target image. The acquisition unit includes a fourth acquisition module and a first detection module: the fourth acquisition module is used to acquire a target image, and the first detection module is used to detect the position information of the seat area based on the target image; The determining unit includes a second detection module and a processing module: the second detection module is used to detect the position information of the face region based on the target image; the processing module is used to determine that the passenger is in the seat if both the position information of the seat region and the position information of the face region are detected, and the relationship between the position information of the seat region and the position information of the face region satisfies a predetermined condition; if the position information of the seat region is detected but the position information of the face region is not detected, or if both the position information of the seat region and the position information of the face region are detected, and the relationship between the position information of the seat region and the position information of the face region does not satisfy the predetermined condition... Under predetermined conditions, the processing module inputs the portion of the target image corresponding to the location information of the seat area into a pre-trained presence judgment model to obtain a judgment result on whether the passenger is in the seat; if neither the location information of the seat area nor the location information of the face area is detected, the processing module uses the location information of the historical seat area as the location information of the seat area, inputs the portion of the target image corresponding to the location information of the seat area into the pre-trained presence judgment model to obtain a judgment result on whether the passenger is in the seat, wherein the location information of the historical seat area is the location information of the seat area determined based on the image of the target image at the previous moment.

14. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the non-volatile storage medium to perform the in-situ state determination method according to any one of claims 1 to 12.

15. A processor, characterized in that, The processor is used to run a program, wherein the program executes the in-situ state determination method according to any one of claims 1 to 12.

Images